2023-12-04T05:18:43.953 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-04T05:18:43.953 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --verbose --archive /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} --name vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi -- /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/orig.config.yaml 2023-12-04T05:18:43.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-04T05:18:44.096 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641 branch: wip-vshankar-testing-20231127.102654 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} email: vshankar@redhat.com first_in_suite: false job_id: '7476641' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20231127.102654 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - client session with non-allowable root sha1: 47c1c547372ed5bff7984647dc200bac0e019caf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 47c1c547372ed5bff7984647dc200bac0e019caf thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231127.102654 sha1: 47c1c547372ed5bff7984647dc200bac0e019caf owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1098 sha1: 47c1c547372ed5bff7984647dc200bac0e019caf sleep_before_teardown: 0 subset: 854/1024 suite: fs suite_branch: wip-vshankar-testing-20231127.102654 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 47c1c547372ed5bff7984647dc200bac0e019caf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP7TOh+HZ3oyc1UE1dEmLMraqDwNhhMUnk6LiwOPFMRjk+UfO70Y+SWF1YQSouzAW6QbWiogrnTHE8H7oEPYl2U= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQEz4rvWx4sxcwW76PgSTUPMID8HIMA8zxtTk/Wv1482QciO4OcN7Sh7KpO2C9zAeVTY+hosTMmRVjHxmy5GEY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_sessionmap teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-04_05:06:51 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-04T05:18:44.096 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa; will attempt to use it 2023-12-04T05:18:44.097 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks 2023-12-04T05:18:44.097 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-04T05:18:44.098 INFO:teuthology.task.internal:Checking packages... 2023-12-04T05:18:44.123 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '47c1c547372ed5bff7984647dc200bac0e019caf' 2023-12-04T05:18:44.123 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-04T05:18:44.123 INFO:teuthology.packaging:ref: None 2023-12-04T05:18:44.123 INFO:teuthology.packaging:tag: None 2023-12-04T05:18:44.123 INFO:teuthology.packaging:branch: wip-vshankar-testing-20231127.102654 2023-12-04T05:18:44.123 INFO:teuthology.packaging:sha1: 47c1c547372ed5bff7984647dc200bac0e019caf 2023-12-04T05:18:44.123 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20231127.102654 2023-12-04T05:18:44.387 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7531.g47c1c547 2023-12-04T05:18:44.387 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-04T05:18:44.396 INFO:teuthology.task.internal:no buildpackages task found 2023-12-04T05:18:44.396 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-04T05:18:44.438 INFO:teuthology.task.internal:Saving configuration 2023-12-04T05:18:44.455 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-04T05:18:44.504 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-04T05:18:44.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-04 05:08:25.501655', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP7TOh+HZ3oyc1UE1dEmLMraqDwNhhMUnk6LiwOPFMRjk+UfO70Y+SWF1YQSouzAW6QbWiogrnTHE8H7oEPYl2U='} 2023-12-04T05:18:44.552 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-04 05:08:25.503205', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQEz4rvWx4sxcwW76PgSTUPMID8HIMA8zxtTk/Wv1482QciO4OcN7Sh7KpO2C9zAeVTY+hosTMmRVjHxmy5GEY='} 2023-12-04T05:18:44.552 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-04T05:18:44.558 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-04T05:18:44.558 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-04T05:18:44.558 INFO:teuthology.run_tasks:Running task console_log... 2023-12-04T05:18:44.673 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe2e8026a60>, signals=[15]) 2023-12-04T05:18:44.673 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-04T05:18:44.688 INFO:teuthology.task.internal:Opening connections... 2023-12-04T05:18:44.688 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:18:44.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:18:44.773 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:18:44.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:18:44.858 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-04T05:18:44.865 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2023-12-04T05:18:44.883 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2023-12-04T05:18:44.883 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2023-12-04T05:18:44.940 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-04T05:18:44.941 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-04T05:18:44.941 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-04T05:18:44.941 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-12-04T05:18:44.968 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2023-12-04T05:18:44.985 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2023-12-04T05:18:44.985 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-04T05:18:45.041 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2023-12-04T05:18:45.042 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-04T05:18:45.042 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-04T05:18:45.042 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-04T05:18:45.042 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-12-04T05:18:45.068 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-04T05:18:45.078 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-04T05:18:45.113 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-04T05:18:45.114 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2023-12-04T05:18:45.116 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2023-12-04T05:18:45.132 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-04T05:18:45.205 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-04T05:18:45.205 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2023-12-04T05:18:45.208 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2023-12-04T05:18:45.360 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-04T05:18:45.419 INFO:teuthology.run_tasks:Running task kernel... 2023-12-04T05:18:45.434 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-04T05:18:45.435 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-04T05:18:45.435 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-04T05:18:45.436 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-04T05:18:45.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:18:45.452 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2023-12-04T05:18:45.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:18:45.453 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2023-12-04T05:18:45.509 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-361.el9.x86_64 2023-12-04T05:18:45.509 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-361.el9.x86_64 2023-12-04T05:18:45.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2023-12-04T05:18:45.511 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-361.el9.x86_64 2023-12-04T05:18:45.514 INFO:teuthology.task.kernel:Running kernel on smithi111: 5.14.0-361.el9.x86_64 2023-12-04T05:18:45.514 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y kernel 2023-12-04T05:18:47.686 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2023-12-04T05:18:47.696 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-12-04T05:18:49.011 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2023-12-04T05:18:49.338 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-9 - BaseOS 3.2 MB/s | 2.0 MB 00:00 2023-12-04T05:18:50.696 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 7.9 MB 00:01 2023-12-04T05:18:51.027 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 7.9 MB 00:01 2023-12-04T05:18:52.897 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 17 MB/s | 18 MB 00:01 2023-12-04T05:18:53.811 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-12-04T05:18:57.222 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 5.8 MB 00:00 2023-12-04T05:18:58.807 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2023-12-04T05:18:59.420 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 5.8 MB 00:02 2023-12-04T05:19:00.908 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2023-12-04T05:19:01.555 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 15 kB 00:00 2023-12-04T05:19:04.930 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 20 MB 00:03 2023-12-04T05:19:05.766 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-12-04T05:19:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-04T05:19:08.115 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-04T05:19:08.220 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-12-04T05:19:08.232 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-12-04T05:19:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2023-12-04T05:19:08.232 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-12-04T05:19:08.232 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2023-12-04T05:19:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: kernel x86_64 5.14.0-388.el9 baseos 5.6 M 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core x86_64 5.14.0-388.el9 baseos 20 M 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules x86_64 5.14.0-388.el9 baseos 38 M 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core x86_64 5.14.0-388.el9 baseos 32 M 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout:Install 4 Packages 2023-12-04T05:19:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:19:08.234 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 96 M 2023-12-04T05:19:08.234 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 121 M 2023-12-04T05:19:08.234 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-12-04T05:19:09.626 INFO:teuthology.orchestra.run.smithi111.stdout:(1/4): kernel-5.14.0-388.el9.x86_64.rpm 4.1 MB/s | 5.6 MB 00:01 2023-12-04T05:19:09.746 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-12-04T05:19:10.251 INFO:teuthology.orchestra.run.smithi111.stdout:(2/4): kernel-core-5.14.0-388.el9.x86_64.rpm 10 MB/s | 20 MB 00:01 2023-12-04T05:19:11.035 INFO:teuthology.orchestra.run.smithi111.stdout:(3/4): kernel-modules-5.14.0-388.el9.x86_64.rpm 14 MB/s | 38 MB 00:02 2023-12-04T05:19:12.104 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-04T05:19:12.104 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-04T05:19:12.197 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-388.el9 baseos 5.6 M 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-388.el9 baseos 20 M 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-388.el9 baseos 38 M 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-388.el9 baseos 32 M 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:19:12.202 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-04T05:19:12.203 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-04T05:19:12.203 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2023-12-04T05:19:12.203 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:19:12.203 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 96 M 2023-12-04T05:19:12.203 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 121 M 2023-12-04T05:19:12.203 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-12-04T05:19:12.268 INFO:teuthology.orchestra.run.smithi111.stdout:(4/4): kernel-modules-core-5.14.0-388.el9.x86_6 12 MB/s | 32 MB 00:02 2023-12-04T05:19:12.268 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-12-04T05:19:12.269 INFO:teuthology.orchestra.run.smithi111.stdout:Total 24 MB/s | 96 MB 00:04 2023-12-04T05:19:13.064 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-388.el9.x86_64.rpm 8.3 MB/s | 5.6 MB 00:00 2023-12-04T05:19:13.121 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-12-04T05:19:13.482 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-388.el9.x86_64.rpm 19 MB/s | 20 MB 00:01 2023-12-04T05:19:13.492 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-12-04T05:19:13.492 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-12-04T05:19:14.789 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-12-04T05:19:14.789 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-12-04T05:19:15.090 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-core-5.14.0-388.el9.x86_6 16 MB/s | 32 MB 00:02 2023-12-04T05:19:15.515 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-5.14.0-388.el9.x86_64.rpm 12 MB/s | 38 MB 00:03 2023-12-04T05:19:15.515 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-12-04T05:19:15.515 INFO:teuthology.orchestra.run.smithi192.stdout:Total 29 MB/s | 96 MB 00:03 2023-12-04T05:19:16.371 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-12-04T05:19:16.744 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-12-04T05:19:16.745 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-12-04T05:19:17.693 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-12-04T05:19:18.063 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-12-04T05:19:18.064 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-12-04T05:19:18.423 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-core-5.14.0-388.el9.x86_64 1/4 2023-12-04T05:19:18.560 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-04T05:19:21.018 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-12-04T05:19:21.332 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-04T05:19:21.521 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-04T05:19:21.649 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-388.el9.x86_64 1/4 2023-12-04T05:19:21.767 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-04T05:19:24.511 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-04T05:19:24.686 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-04T05:19:26.124 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-04T05:19:26.337 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:19:29.318 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-04T05:19:29.518 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:19:30.994 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:19:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:19:59.708 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:19:59.751 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:00.209 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:00.209 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-5.14.0-388.el9.x86_64 1/4 2023-12-04T05:20:00.209 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-04T05:20:00.209 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-5.14.0-388.el9.x86_64 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core-5.14.0-388.el9.x86_64 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-5.14.0-388.el9.x86_64 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core-5.14.0-388.el9.x86_64 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:20:00.631 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-12-04T05:20:01.119 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2023-12-04T05:20:01.278 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-04T05:20:01.278 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-04T05:20:01.278 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-388.el9.x86_64 2023-12-04T05:20:01.279 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-04T05:20:01.279 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-04T05:20:01.279 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-04T05:20:01.279 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2023-12-04T05:20:01.279 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2023-12-04T05:20:01.395 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-04T05:20:01.395 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-04T05:20:01.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2023-12-04T05:20:01.438 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 68.el9 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8264254 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-04T05:20:01.439 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:20:01.440 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2023-12-04T05:20:01.441 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-04T05:20:01.441 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 5.14.0-388.el9.x86_64 2023-12-04T05:20:01.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-04T05:20:02.111 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2023-12-04T05:20:03.328 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:03.371 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:03.476 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-04T05:20:03.516 INFO:teuthology.orchestra.run.smithi111.stderr:done 2023-12-04T05:20:03.519 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-12-04T05:20:03.535 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.qh8L6wcQwq 2023-12-04T05:20:03.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qh8L6wcQwq 2023-12-04T05:20:03.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.qh8L6wcQwq 2023-12-04T05:20:03.744 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.qh8L6wcQwq is 6KB 2023-12-04T05:20:03.793 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.qh8L6wcQwq 2023-12-04T05:20:03.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2023-12-04T05:20:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-388.el9.x86_64 1/4 2023-12-04T05:20:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-04T05:20:03.818 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-04T05:20:03.882 INFO:teuthology.orchestra.run.smithi111.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-04T05:20:03.883 INFO:teuthology.orchestra.run.smithi111.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-04T05:20:03.883 INFO:teuthology.orchestra.run.smithi111.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-04T05:20:03.883 INFO:teuthology.orchestra.run.smithi111.stdout:15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64.conf 2023-12-04T05:20:03.883 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64 2023-12-04T05:20:04.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2023-12-04T05:20:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-04T05:20:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:20:04.230 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-12-04T05:20:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-388.el9.x86_64 2023-12-04T05:20:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-388.el9.x86_64 2023-12-04T05:20:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-388.el9.x86_64 2023-12-04T05:20:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-388.el9.x86_64 2023-12-04T05:20:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:20:04.231 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-04T05:20:04.754 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2023-12-04T05:20:04.902 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-04T05:20:04.903 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-04T05:20:04.903 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-388.el9.x86_64 2023-12-04T05:20:04.903 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-04T05:20:04.903 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-04T05:20:04.903 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-12-04T05:20:04.903 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2023-12-04T05:20:04.903 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2023-12-04T05:20:05.021 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-04T05:20:05.021 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-04T05:20:05.022 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2023-12-04T05:20:05.055 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2023-12-04T05:20:05.055 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2023-12-04T05:20:05.056 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:20:05.057 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2023-12-04T05:20:05.058 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-04T05:20:05.058 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-388.el9.x86_64 2023-12-04T05:20:05.058 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-04T05:20:05.668 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2023-12-04T05:20:07.047 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-04T05:20:07.084 INFO:teuthology.orchestra.run.smithi192.stderr:done 2023-12-04T05:20:07.092 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-04T05:20:07.107 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.2zQopaWIPv 2023-12-04T05:20:07.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2zQopaWIPv 2023-12-04T05:20:07.179 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.2zQopaWIPv 2023-12-04T05:20:07.318 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.2zQopaWIPv is 6KB 2023-12-04T05:20:07.339 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.2zQopaWIPv 2023-12-04T05:20:07.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2023-12-04T05:20:07.418 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-04T05:20:07.418 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-04T05:20:07.418 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-04T05:20:07.418 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64.conf 2023-12-04T05:20:07.419 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64 2023-12-04T05:20:07.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2023-12-04T05:20:34.059 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-04T05:20:34.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:20:34.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:20:37.582 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-04T05:20:37.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-04T05:20:37.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:20:52.578 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-12-04T05:20:55.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2023-12-04T05:21:01.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:21:01.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:21:04.642 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-12-04T05:21:04.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-04T05:21:04.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:21:07.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2023-12-04T05:21:16.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:21:16.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:21:19.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-12-04T05:21:19.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-04T05:21:19.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:21:34.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:21:34.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:21:37.794 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-12-04T05:21:51.880 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-12-04T05:21:52.235 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-04T05:21:52.235 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-388.el9.x86_64"... 2023-12-04T05:21:52.235 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2023-12-04T05:21:52.251 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-388.el9.x86_64 2023-12-04T05:21:52.251 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-388.el9.x86_64 vs 5.14.0-388.el9.x86_64 2023-12-04T05:21:52.251 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-04T05:21:52.251 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-04T05:21:53.252 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-04T05:21:53.252 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-04T05:21:53.344 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2023-12-04T05:21:53.387 DEBUG:teuthology.parallel:result is None 2023-12-04T05:21:55.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:21:55.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:21:57.839 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-12-04T05:21:58.310 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:21:58.310 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-388.el9.x86_64"... 2023-12-04T05:21:58.310 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2023-12-04T05:21:58.325 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-388.el9.x86_64 2023-12-04T05:21:58.325 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-388.el9.x86_64 vs 5.14.0-388.el9.x86_64 2023-12-04T05:21:58.325 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-04T05:21:58.325 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-04T05:21:59.327 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-04T05:21:59.327 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-04T05:21:59.426 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2023-12-04T05:21:59.471 DEBUG:teuthology.parallel:result is None 2023-12-04T05:21:59.472 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-04T05:21:59.479 INFO:teuthology.task.internal:Creating test directory... 2023-12-04T05:21:59.480 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-04T05:21:59.482 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-04T05:21:59.501 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-04T05:21:59.509 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-04T05:21:59.545 INFO:teuthology.task.internal:Creating archive directory... 2023-12-04T05:21:59.546 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-04T05:21:59.549 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-04T05:21:59.577 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-04T05:21:59.600 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-04T05:21:59.600 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-04T05:21:59.612 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-04T05:21:59.647 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-04T05:21:59.648 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-04T05:21:59.661 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-04T05:21:59.662 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-04T05:21:59.663 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-04T05:21:59.671 INFO:teuthology.task.internal:Configuring sudo... 2023-12-04T05:21:59.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-04T05:21:59.706 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-04T05:21:59.735 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-04T05:21:59.745 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-04T05:21:59.746 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-04T05:21:59.773 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-04T05:21:59.791 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-04T05:21:59.871 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-04T05:21:59.932 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-04T05:22:00.015 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-04T05:22:00.057 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:22:00.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-04T05:22:00.128 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-04T05:22:00.173 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-04T05:22:00.235 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-04T05:22:00.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-04T05:22:00.372 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:22:00.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-04T05:22:00.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2023-12-04T05:22:00.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2023-12-04T05:22:00.495 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-04T05:22:00.563 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-04T05:22:01.032 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-04T05:22:01.041 INFO:teuthology.task.internal:Starting timer... 2023-12-04T05:22:01.041 INFO:teuthology.run_tasks:Running task pcp... 2023-12-04T05:22:01.074 INFO:teuthology.run_tasks:Running task selinux... 2023-12-04T05:22:01.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2023-12-04T05:22:01.166 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2023-12-04T05:22:01.169 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2023-12-04T05:22:01.219 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2023-12-04T05:22:01.227 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-04T05:22:01.227 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2023-12-04T05:22:01.247 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2023-12-04T05:22:01.247 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2023-12-04T05:22:01.272 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2023-12-04T05:22:01.273 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2023-12-04T05:22:01.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-04T05:22:01.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:22:01.314 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-04T05:22:01.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:22:01.344 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-04T05:22:01.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2023-12-04T05:22:01.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2023-12-04T05:22:01.424 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-04T05:22:01.439 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-04T05:22:01.439 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-04T05:22:01.504 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-04T05:22:01.505 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi111.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-04T05:35:47.464 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2023-12-04T05:35:47.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:35:47.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:35:47.587 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-12-04T05:35:47.612 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-04T05:35:47.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-04T05:35:47.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-04T05:35:47.701 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-12-04T05:35:47.767 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-04T05:35:47.768 INFO:teuthology.run_tasks:Running task clock... 2023-12-04T05:35:47.781 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-04T05:35:47.781 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-04T05:35:47.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-04T05:35:47.785 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-04T05:35:47.785 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-04T05:35:47.812 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-04T05:35:47.831 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-04T05:35:47.841 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-04T05:35:47.858 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2023-12-04T05:35:47.859 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-04T05:35:47.876 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2023-12-04T05:35:47.888 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-04T05:35:47.894 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2023-12-04T05:35:47.903 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2023-12-04T05:35:47.904 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-04T05:35:47.921 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-04T05:35:47.937 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-04T05:35:47.947 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2023-12-04T05:35:47.950 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-04T05:35:47.950 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-12-04T05:35:47.950 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.950 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.950 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.950 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.979 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2023-12-04T05:35:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-04T05:35:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-12-04T05:35:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.983 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:35:47.983 INFO:teuthology.run_tasks:Running task install... 2023-12-04T05:35:47.998 DEBUG:teuthology.task.install:project ceph 2023-12-04T05:35:47.999 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}} 2023-12-04T05:35:47.999 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'} 2023-12-04T05:35:47.999 INFO:teuthology.task.install:Using flavor: default 2023-12-04T05:35:48.013 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-04T05:35:48.014 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-04T05:35:48.014 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'tag': None, 'wait_for_package': False} 2023-12-04T05:35:48.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-12-04T05:35:48.017 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'tag': None, 'wait_for_package': False} 2023-12-04T05:35:48.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-12-04T05:35:48.172 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/ 2023-12-04T05:35:48.172 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7531.g47c1c547 2023-12-04T05:35:48.249 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/ 2023-12-04T05:35:48.249 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7531.g47c1c547 2023-12-04T05:35:48.303 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-04T05:35:48.303 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:35:48.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-04T05:35:48.332 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-04T05:35:48.332 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/47c1c547372ed5bff7984647dc200bac0e019caf/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-04T05:35:48.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-04T05:35:48.417 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654/47c1c547372ed5bff7984647dc200bac0e019caf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-04T05:35:48.417 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:35:48.417 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-04T05:35:48.446 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-04T05:35:48.446 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/47c1c547372ed5bff7984647dc200bac0e019caf/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-04T05:35:48.483 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-04T05:35:48.519 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-04T05:35:48.551 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2023-12-04T05:35:48.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2023-12-04T05:35:48.601 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-04T05:35:48.670 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2023-12-04T05:35:48.670 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2023-12-04T05:35:48.798 INFO:teuthology.orchestra.run.smithi111.stdout:73 files removed 2023-12-04T05:35:48.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-04T05:35:48.926 INFO:teuthology.orchestra.run.smithi192.stdout:73 files removed 2023-12-04T05:35:48.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-04T05:35:49.549 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 192 kB/s | 78 kB 00:00 2023-12-04T05:35:49.676 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 194 kB/s | 78 kB 00:00 2023-12-04T05:35:49.858 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 37 kB/s | 9.5 kB 00:00 2023-12-04T05:35:50.099 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-12-04T05:35:50.175 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2023-12-04T05:35:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2023-12-04T05:35:50.562 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-9 - AppStream 18 MB/s | 6.4 MB 00:00 2023-12-04T05:35:50.947 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 17 MB/s | 6.4 MB 00:00 2023-12-04T05:35:51.920 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2023-12-04T05:35:52.381 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2023-12-04T05:35:54.421 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 7.9 MB 00:01 2023-12-04T05:35:56.389 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 21 MB/s | 18 MB 00:00 2023-12-04T05:36:01.082 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 5.8 MB 00:01 2023-12-04T05:36:27.559 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 229 kB/s | 7.9 MB 00:35 2023-12-04T05:36:30.617 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 18 MB 00:01 2023-12-04T05:36:35.192 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 5.8 MB 00:01 2023-12-04T05:36:35.728 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 454 B/s | 15 kB 00:33 2023-12-04T05:36:36.826 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2023-12-04T05:36:37.196 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2023-12-04T05:36:38.536 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-12-04T05:36:41.863 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-12-04T05:36:43.403 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-12-04T05:36:44.540 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-04T05:36:44.540 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-04T05:36:44.552 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-04T05:36:44.553 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-04T05:36:44.553 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-04T05:36:44.553 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-04T05:36:44.555 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-04T05:36:44.557 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-04T05:36:44.558 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-04T05:36:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-04T05:36:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2023-12-04T05:36:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-12-04T05:36:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2023-12-04T05:36:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2023-12-04T05:36:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-04T05:36:44.664 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 6.6 k 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 5.2 M 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 859 k 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 141 k 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 1.5 M 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 133 k 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 1.8 M 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 7.4 M 2023-12-04T05:36:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 50 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 8.7 M 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 41 M 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 262 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 773 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 217 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 18 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-04T05:36:44.666 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 31 k 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 719 k 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 126 k 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-04T05:36:44.667 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 162 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 321 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 298 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 99 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 85 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 3.0 M 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 170 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-04T05:36:44.668 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 3.3 M 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 3.0 M 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-04T05:36:44.669 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 19 M 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 28 k 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 2.1 M 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 243 k 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 4.4 M 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 18 M 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 15 k 2023-12-04T05:36:44.670 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 24 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 163 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-04T05:36:44.671 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 476 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 4.8 M 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-04T05:36:44.672 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 45 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 129 k 2023-12-04T05:36:44.673 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-04T05:36:44.674 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-04T05:36:44.675 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-04T05:36:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-04T05:36:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-04T05:36:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-04T05:36:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-04T05:36:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-04T05:36:44.676 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-04T05:36:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-04T05:36:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-04T05:36:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-04T05:36:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-04T05:36:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-04T05:36:44.677 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-04T05:36:44.678 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-04T05:36:44.679 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-04T05:36:44.680 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:Install 135 Packages 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 15 Packages 2023-12-04T05:36:44.681 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:36:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 205 M 2023-12-04T05:36:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-12-04T05:36:45.298 INFO:teuthology.orchestra.run.smithi192.stdout:(1/150): ceph-18.0.0-7531.g47c1c547.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2023-12-04T05:36:45.573 INFO:teuthology.orchestra.run.smithi192.stdout:(2/150): ceph-base-18.0.0-7531.g47c1c547.el9.x8 10 MB/s | 5.2 MB 00:00 2023-12-04T05:36:45.623 INFO:teuthology.orchestra.run.smithi192.stdout:(3/150): ceph-immutable-object-cache-18.0.0-753 2.8 MB/s | 141 kB 00:00 2023-12-04T05:36:45.782 INFO:teuthology.orchestra.run.smithi192.stdout:(4/150): ceph-mds-18.0.0-7531.g47c1c547.el9.x86 13 MB/s | 2.1 MB 00:00 2023-12-04T05:36:45.815 INFO:teuthology.orchestra.run.smithi192.stdout:(5/150): ceph-fuse-18.0.0-7531.g47c1c547.el9.x8 1.6 MB/s | 859 kB 00:00 2023-12-04T05:36:45.891 INFO:teuthology.orchestra.run.smithi192.stdout:(6/150): ceph-mgr-18.0.0-7531.g47c1c547.el9.x86 13 MB/s | 1.5 MB 00:00 2023-12-04T05:36:46.107 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-04T05:36:46.107 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-04T05:36:46.121 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-04T05:36:46.121 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-04T05:36:46.122 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-04T05:36:46.124 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-04T05:36:46.126 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-04T05:36:46.129 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-04T05:36:46.130 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-04T05:36:46.204 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================ 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================ 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 6.6 k 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 5.2 M 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 859 k 2023-12-04T05:36:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 141 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 1.5 M 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 133 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 1.8 M 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 7.4 M 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 50 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 8.7 M 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 41 M 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 262 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 773 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 217 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 18 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-04T05:36:46.222 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 31 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 719 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 126 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 162 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 321 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 298 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 99 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 85 k 2023-12-04T05:36:46.223 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 3.0 M 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 170 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 3.3 M 2023-12-04T05:36:46.224 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 3.0 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 19 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 28 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 2.1 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 243 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 4.4 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 18 M 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7531.g47c1c547.el9 ceph-noarch 15 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 24 k 2023-12-04T05:36:46.225 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 163 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 476 k 2023-12-04T05:36:46.226 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 4.8 M 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-04T05:36:46.227 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 45 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:18.0.0-7531.g47c1c547.el9 ceph 129 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-04T05:36:46.228 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-04T05:36:46.229 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-04T05:36:46.230 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-04T05:36:46.231 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================ 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Install 135 Packages 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 15 Packages 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 205 M 2023-12-04T05:36:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-12-04T05:36:46.833 INFO:teuthology.orchestra.run.smithi192.stdout:(7/150): ceph-osd-18.0.0-7531.g47c1c547.el9.x86 19 MB/s | 18 MB 00:00 2023-12-04T05:36:47.263 INFO:teuthology.orchestra.run.smithi111.stdout:(1/150): ceph-18.0.0-7531.g47c1c547.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2023-12-04T05:36:47.376 INFO:teuthology.orchestra.run.smithi192.stdout:(8/150): ceph-mon-18.0.0-7531.g47c1c547.el9.x86 2.8 MB/s | 4.4 MB 00:01 2023-12-04T05:36:47.475 INFO:teuthology.orchestra.run.smithi192.stdout:(9/150): ceph-radosgw-18.0.0-7531.g47c1c547.el9 14 MB/s | 8.7 MB 00:00 2023-12-04T05:36:47.492 INFO:teuthology.orchestra.run.smithi192.stdout:(10/150): ceph-selinux-18.0.0-7531.g47c1c547.el 206 kB/s | 24 kB 00:00 2023-12-04T05:36:47.946 INFO:teuthology.orchestra.run.smithi111.stdout:(2/150): ceph-fuse-18.0.0-7531.g47c1c547.el9.x8 1.2 MB/s | 859 kB 00:00 2023-12-04T05:36:48.121 INFO:teuthology.orchestra.run.smithi111.stdout:(3/150): ceph-immutable-object-cache-18.0.0-753 806 kB/s | 141 kB 00:00 2023-12-04T05:36:48.197 INFO:teuthology.orchestra.run.smithi111.stdout:(4/150): ceph-base-18.0.0-7531.g47c1c547.el9.x8 3.9 MB/s | 5.2 MB 00:01 2023-12-04T05:36:48.325 INFO:teuthology.orchestra.run.smithi192.stdout:(11/150): cephfs-mirror-18.0.0-7531.g47c1c547.e 260 kB/s | 217 kB 00:00 2023-12-04T05:36:48.418 INFO:teuthology.orchestra.run.smithi192.stdout:(12/150): libcephfs-devel-18.0.0-7531.g47c1c547 339 kB/s | 31 kB 00:00 2023-12-04T05:36:48.481 INFO:teuthology.orchestra.run.smithi111.stdout:(5/150): ceph-mgr-18.0.0-7531.g47c1c547.el9.x86 5.2 MB/s | 1.5 MB 00:00 2023-12-04T05:36:48.793 INFO:teuthology.orchestra.run.smithi192.stdout:(13/150): libcephfs2-18.0.0-7531.g47c1c547.el9. 1.9 MB/s | 719 kB 00:00 2023-12-04T05:36:48.902 INFO:teuthology.orchestra.run.smithi192.stdout:(14/150): libcephsqlite-18.0.0-7531.g47c1c547.e 1.5 MB/s | 163 kB 00:00 2023-12-04T05:36:49.002 INFO:teuthology.orchestra.run.smithi192.stdout:(15/150): librados-devel-18.0.0-7531.g47c1c547. 1.2 MB/s | 126 kB 00:00 2023-12-04T05:36:49.197 INFO:teuthology.orchestra.run.smithi111.stdout:(6/150): ceph-mon-18.0.0-7531.g47c1c547.el9.x86 6.1 MB/s | 4.4 MB 00:00 2023-12-04T05:36:49.235 INFO:teuthology.orchestra.run.smithi192.stdout:(16/150): libradosstriper1-18.0.0-7531.g47c1c54 2.0 MB/s | 476 kB 00:00 2023-12-04T05:36:49.594 INFO:teuthology.orchestra.run.smithi192.stdout:(17/150): ceph-common-18.0.0-7531.g47c1c547.el9 4.2 MB/s | 19 MB 00:04 2023-12-04T05:36:49.640 INFO:teuthology.orchestra.run.smithi111.stdout:(7/150): ceph-mds-18.0.0-7531.g47c1c547.el9.x86 1.4 MB/s | 2.1 MB 00:01 2023-12-04T05:36:49.669 INFO:teuthology.orchestra.run.smithi192.stdout:(18/150): python3-ceph-argparse-18.0.0-7531.g47 604 kB/s | 45 kB 00:00 2023-12-04T05:36:49.778 INFO:teuthology.orchestra.run.smithi192.stdout:(19/150): python3-ceph-common-18.0.0-7531.g47c1 1.2 MB/s | 129 kB 00:00 2023-12-04T05:36:49.878 INFO:teuthology.orchestra.run.smithi192.stdout:(20/150): python3-cephfs-18.0.0-7531.g47c1c547. 1.6 MB/s | 162 kB 00:00 2023-12-04T05:36:49.987 INFO:teuthology.orchestra.run.smithi192.stdout:(21/150): python3-rados-18.0.0-7531.g47c1c547.e 2.9 MB/s | 321 kB 00:00 2023-12-04T05:36:50.095 INFO:teuthology.orchestra.run.smithi192.stdout:(22/150): python3-rbd-18.0.0-7531.g47c1c547.el9 2.7 MB/s | 298 kB 00:00 2023-12-04T05:36:50.196 INFO:teuthology.orchestra.run.smithi192.stdout:(23/150): python3-rgw-18.0.0-7531.g47c1c547.el9 994 kB/s | 99 kB 00:00 2023-12-04T05:36:50.296 INFO:teuthology.orchestra.run.smithi192.stdout:(24/150): rbd-fuse-18.0.0-7531.g47c1c547.el9.x8 852 kB/s | 85 kB 00:00 2023-12-04T05:36:50.921 INFO:teuthology.orchestra.run.smithi192.stdout:(25/150): rbd-mirror-18.0.0-7531.g47c1c547.el9. 4.8 MB/s | 3.0 MB 00:00 2023-12-04T05:36:51.014 INFO:teuthology.orchestra.run.smithi192.stdout:(26/150): rbd-nbd-18.0.0-7531.g47c1c547.el9.x86 1.8 MB/s | 170 kB 00:00 2023-12-04T05:36:51.356 INFO:teuthology.orchestra.run.smithi111.stdout:(8/150): ceph-osd-18.0.0-7531.g47c1c547.el9.x86 8.2 MB/s | 18 MB 00:02 2023-12-04T05:36:51.422 INFO:teuthology.orchestra.run.smithi192.stdout:(27/150): ceph-test-18.0.0-7531.g47c1c547.el9.x 10 MB/s | 41 MB 00:03 2023-12-04T05:36:51.423 INFO:teuthology.orchestra.run.smithi111.stdout:(9/150): ceph-selinux-18.0.0-7531.g47c1c547.el9 363 kB/s | 24 kB 00:00 2023-12-04T05:36:51.447 INFO:teuthology.orchestra.run.smithi192.stdout:(28/150): ceph-grafana-dashboards-18.0.0-7531.g 65 kB/s | 28 kB 00:00 2023-12-04T05:36:51.464 INFO:teuthology.orchestra.run.smithi192.stdout:(29/150): ceph-mgr-cephadm-18.0.0-7531.g47c1c54 3.1 MB/s | 133 kB 00:00 2023-12-04T05:36:51.540 INFO:teuthology.orchestra.run.smithi192.stdout:(30/150): librgw2-18.0.0-7531.g47c1c547.el9.x86 2.1 MB/s | 4.8 MB 00:02 2023-12-04T05:36:51.715 INFO:teuthology.orchestra.run.smithi192.stdout:(31/150): ceph-mgr-modules-core-18.0.0-7531.g47 1.4 MB/s | 243 kB 00:00 2023-12-04T05:36:51.823 INFO:teuthology.orchestra.run.smithi192.stdout:(32/150): ceph-mgr-rook-18.0.0-7531.g47c1c547.e 464 kB/s | 50 kB 00:00 2023-12-04T05:36:51.861 INFO:teuthology.orchestra.run.smithi192.stdout:(33/150): ceph-mgr-dashboard-18.0.0-7531.g47c1c 4.3 MB/s | 1.8 MB 00:00 2023-12-04T05:36:51.903 INFO:teuthology.orchestra.run.smithi192.stdout:(34/150): ceph-prometheus-alerts-18.0.0-7531.g4 191 kB/s | 15 kB 00:00 2023-12-04T05:36:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:(35/150): ceph-volume-18.0.0-7531.g47c1c547.el9 2.8 MB/s | 262 kB 00:00 2023-12-04T05:36:52.037 INFO:teuthology.orchestra.run.smithi192.stdout:(36/150): cephfs-top-18.0.0-7531.g47c1c547.el9. 219 kB/s | 18 kB 00:00 2023-12-04T05:36:52.178 INFO:teuthology.orchestra.run.smithi192.stdout:(37/150): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2023-12-04T05:36:52.237 INFO:teuthology.orchestra.run.smithi192.stdout:(38/150): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2023-12-04T05:36:52.271 INFO:teuthology.orchestra.run.smithi192.stdout:(39/150): cephadm-18.0.0-7531.g47c1c547.el9.noa 2.1 MB/s | 773 kB 00:00 2023-12-04T05:36:52.298 INFO:teuthology.orchestra.run.smithi192.stdout:(40/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2023-12-04T05:36:52.321 INFO:teuthology.orchestra.run.smithi192.stdout:(41/150): elfutils-libelf-devel-0.190-1.el9.x86 431 kB/s | 21 kB 00:00 2023-12-04T05:36:52.348 INFO:teuthology.orchestra.run.smithi192.stdout:(42/150): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-12-04T05:36:52.372 INFO:teuthology.orchestra.run.smithi192.stdout:(43/150): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2023-12-04T05:36:52.464 INFO:teuthology.orchestra.run.smithi192.stdout:(44/150): ceph-mgr-diskprediction-local-18.0.0- 7.4 MB/s | 7.4 MB 00:00 2023-12-04T05:36:52.547 INFO:teuthology.orchestra.run.smithi192.stdout:(45/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2023-12-04T05:36:52.572 INFO:teuthology.orchestra.run.smithi192.stdout:(46/150): flexiblas-openblas-openmp-3.0.4-8.el9 87 kB/s | 17 kB 00:00 2023-12-04T05:36:52.598 INFO:teuthology.orchestra.run.smithi192.stdout:(47/150): jq-1.6-16.el9.x86_64.rpm 1.4 MB/s | 186 kB 00:00 2023-12-04T05:36:52.623 INFO:teuthology.orchestra.run.smithi192.stdout:(48/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-12-04T05:36:52.648 INFO:teuthology.orchestra.run.smithi192.stdout:(49/150): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 10 kB 00:00 2023-12-04T05:36:52.674 INFO:teuthology.orchestra.run.smithi192.stdout:(50/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2023-12-04T05:36:52.707 INFO:teuthology.orchestra.run.smithi192.stdout:(51/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-12-04T05:36:52.732 INFO:teuthology.orchestra.run.smithi192.stdout:(52/150): libuuid-devel-2.37.4-15.el9.x86_64.rp 427 kB/s | 24 kB 00:00 2023-12-04T05:36:52.758 INFO:teuthology.orchestra.run.smithi192.stdout:(53/150): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2023-12-04T05:36:52.774 INFO:teuthology.orchestra.run.smithi192.stdout:(54/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 692 kB/s | 46 kB 00:00 2023-12-04T05:36:52.800 INFO:teuthology.orchestra.run.smithi192.stdout:(55/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-12-04T05:36:52.825 INFO:teuthology.orchestra.run.smithi192.stdout:(56/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-12-04T05:36:52.850 INFO:teuthology.orchestra.run.smithi192.stdout:(57/150): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2023-12-04T05:36:52.867 INFO:teuthology.orchestra.run.smithi192.stdout:(58/150): openblas-0.3.21-2.el9.x86_64.rpm 513 kB/s | 34 kB 00:00 2023-12-04T05:36:52.984 INFO:teuthology.orchestra.run.smithi192.stdout:(59/150): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2023-12-04T05:36:53.026 INFO:teuthology.orchestra.run.smithi192.stdout:(60/150): postgresql-13.11-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2023-12-04T05:36:53.051 INFO:teuthology.orchestra.run.smithi192.stdout:(61/150): postgresql-contrib-13.11-1.el9.x86_64 4.4 MB/s | 811 kB 00:00 2023-12-04T05:36:53.076 INFO:teuthology.orchestra.run.smithi192.stdout:(62/150): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2023-12-04T05:36:53.132 INFO:teuthology.orchestra.run.smithi111.stdout:(10/150): ceph-common-18.0.0-7531.g47c1c547.el9 3.0 MB/s | 19 MB 00:06 2023-12-04T05:36:53.240 INFO:teuthology.orchestra.run.smithi111.stdout:(11/150): cephfs-mirror-18.0.0-7531.g47c1c547.e 2.0 MB/s | 217 kB 00:00 2023-12-04T05:36:53.260 INFO:teuthology.orchestra.run.smithi192.stdout:(63/150): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2023-12-04T05:36:53.316 INFO:teuthology.orchestra.run.smithi111.stdout:(12/150): libcephfs-devel-18.0.0-7531.g47c1c547 415 kB/s | 31 kB 00:00 2023-12-04T05:36:53.327 INFO:teuthology.orchestra.run.smithi192.stdout:(64/150): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2023-12-04T05:36:53.352 INFO:teuthology.orchestra.run.smithi192.stdout:(65/150): python3-jinja2-2.11.3-4.el9.noarch.rp 832 kB/s | 228 kB 00:00 2023-12-04T05:36:53.369 INFO:teuthology.orchestra.run.smithi192.stdout:(66/150): python3-jsonpatch-1.21-16.el9.noarch. 227 kB/s | 24 kB 00:00 2023-12-04T05:36:53.386 INFO:teuthology.orchestra.run.smithi192.stdout:(67/150): python3-jsonpointer-2.0-4.el9.noarch. 540 kB/s | 18 kB 00:00 2023-12-04T05:36:53.411 INFO:teuthology.orchestra.run.smithi192.stdout:(68/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2023-12-04T05:36:53.428 INFO:teuthology.orchestra.run.smithi192.stdout:(69/150): python3-markupsafe-1.1.1-12.el9.x86_6 539 kB/s | 32 kB 00:00 2023-12-04T05:36:53.453 INFO:teuthology.orchestra.run.smithi192.stdout:(70/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2023-12-04T05:36:53.516 INFO:teuthology.orchestra.run.smithi111.stdout:(13/150): ceph-radosgw-18.0.0-7531.g47c1c547.el 2.2 MB/s | 8.7 MB 00:03 2023-12-04T05:36:53.550 INFO:teuthology.orchestra.run.smithi111.stdout:(14/150): libcephfs2-18.0.0-7531.g47c1c547.el9. 3.0 MB/s | 719 kB 00:00 2023-12-04T05:36:53.553 INFO:teuthology.orchestra.run.smithi192.stdout:(71/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2023-12-04T05:36:53.579 INFO:teuthology.orchestra.run.smithi192.stdout:(72/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-12-04T05:36:53.596 INFO:teuthology.orchestra.run.smithi192.stdout:(73/150): python3-prettytable-0.7.2-27.el9.noar 294 kB/s | 41 kB 00:00 2023-12-04T05:36:53.608 INFO:teuthology.orchestra.run.smithi111.stdout:(15/150): librados-devel-18.0.0-7531.g47c1c547. 2.1 MB/s | 126 kB 00:00 2023-12-04T05:36:53.612 INFO:teuthology.orchestra.run.smithi192.stdout:(74/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2023-12-04T05:36:53.633 INFO:teuthology.orchestra.run.smithi111.stdout:(16/150): libcephsqlite-18.0.0-7531.g47c1c547.e 1.4 MB/s | 163 kB 00:00 2023-12-04T05:36:53.638 INFO:teuthology.orchestra.run.smithi192.stdout:(75/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.0 MB/s | 211 kB 00:00 2023-12-04T05:36:53.655 INFO:teuthology.orchestra.run.smithi192.stdout:(76/150): python3-pytz-2021.1-5.el9.noarch.rpm 811 kB/s | 47 kB 00:00 2023-12-04T05:36:53.672 INFO:teuthology.orchestra.run.smithi192.stdout:(77/150): python3-requests-oauthlib-1.3.0-12.el 732 kB/s | 43 kB 00:00 2023-12-04T05:36:53.722 INFO:teuthology.orchestra.run.smithi192.stdout:(78/150): python3-toml-0.10.2-6.el9.noarch.rpm 579 kB/s | 39 kB 00:00 2023-12-04T05:36:53.775 INFO:teuthology.orchestra.run.smithi111.stdout:(17/150): libradosstriper1-18.0.0-7531.g47c1c54 2.8 MB/s | 476 kB 00:00 2023-12-04T05:36:53.855 INFO:teuthology.orchestra.run.smithi192.stdout:(79/150): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 300 kB 00:00 2023-12-04T05:36:53.859 INFO:teuthology.orchestra.run.smithi111.stdout:(18/150): python3-ceph-argparse-18.0.0-7531.g47 542 kB/s | 45 kB 00:00 2023-12-04T05:36:53.945 INFO:teuthology.orchestra.run.smithi111.stdout:(19/150): python3-ceph-common-18.0.0-7531.g47c1 1.5 MB/s | 129 kB 00:00 2023-12-04T05:36:54.014 INFO:teuthology.orchestra.run.smithi192.stdout:(80/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2023-12-04T05:36:54.029 INFO:teuthology.orchestra.run.smithi111.stdout:(20/150): python3-cephfs-18.0.0-7531.g47c1c547. 1.9 MB/s | 162 kB 00:00 2023-12-04T05:36:54.031 INFO:teuthology.orchestra.run.smithi192.stdout:(81/150): uuid-1.6.2-55.el9.x86_64.rpm 184 kB/s | 56 kB 00:00 2023-12-04T05:36:54.056 INFO:teuthology.orchestra.run.smithi192.stdout:(82/150): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 429 kB/s | 85 kB 00:00 2023-12-04T05:36:54.081 INFO:teuthology.orchestra.run.smithi192.stdout:(83/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-12-04T05:36:54.099 INFO:teuthology.orchestra.run.smithi192.stdout:(84/150): zlib-devel-1.2.11-40.el9.x86_64.rpm 662 kB/s | 44 kB 00:00 2023-12-04T05:36:54.124 INFO:teuthology.orchestra.run.smithi192.stdout:(85/150): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2023-12-04T05:36:54.154 INFO:teuthology.orchestra.run.smithi111.stdout:(21/150): python3-rados-18.0.0-7531.g47c1c547.e 2.5 MB/s | 321 kB 00:00 2023-12-04T05:36:54.157 INFO:teuthology.orchestra.run.smithi192.stdout:(86/150): libgfortran-11.4.1-2.3.el9.x86_64.rpm 13 MB/s | 803 kB 00:00 2023-12-04T05:36:54.182 INFO:teuthology.orchestra.run.smithi192.stdout:(87/150): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2023-12-04T05:36:54.208 INFO:teuthology.orchestra.run.smithi192.stdout:(88/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2023-12-04T05:36:54.241 INFO:teuthology.orchestra.run.smithi192.stdout:(89/150): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2023-12-04T05:36:54.266 INFO:teuthology.orchestra.run.smithi192.stdout:(90/150): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-12-04T05:36:54.271 INFO:teuthology.orchestra.run.smithi111.stdout:(22/150): python3-rbd-18.0.0-7531.g47c1c547.el9 2.5 MB/s | 298 kB 00:00 2023-12-04T05:36:54.294 INFO:teuthology.orchestra.run.smithi192.stdout:(91/150): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2023-12-04T05:36:54.317 INFO:teuthology.orchestra.run.smithi192.stdout:(92/150): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2023-12-04T05:36:54.342 INFO:teuthology.orchestra.run.smithi192.stdout:(93/150): python3-urllib3-1.26.5-4.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2023-12-04T05:36:54.501 INFO:teuthology.orchestra.run.smithi192.stdout:(94/150): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2023-12-04T05:36:54.618 INFO:teuthology.orchestra.run.smithi192.stdout:(95/150): openssl-devel-3.0.7-25.el9.x86_64.rpm 13 MB/s | 4.1 MB 00:00 2023-12-04T05:36:54.663 INFO:teuthology.orchestra.run.smithi111.stdout:(23/150): ceph-test-18.0.0-7531.g47c1c547.el9.x 13 MB/s | 41 MB 00:03 2023-12-04T05:36:54.668 INFO:teuthology.orchestra.run.smithi192.stdout:(96/150): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2023-12-04T05:36:54.688 INFO:teuthology.orchestra.run.smithi111.stdout:(24/150): python3-rgw-18.0.0-7531.g47c1c547.el9 238 kB/s | 99 kB 00:00 2023-12-04T05:36:54.722 INFO:teuthology.orchestra.run.smithi111.stdout:(25/150): rbd-fuse-18.0.0-7531.g47c1c547.el9.x8 1.4 MB/s | 85 kB 00:00 2023-12-04T05:36:54.781 INFO:teuthology.orchestra.run.smithi111.stdout:(26/150): rbd-nbd-18.0.0-7531.g47c1c547.el9.x86 2.8 MB/s | 170 kB 00:00 2023-12-04T05:36:54.840 INFO:teuthology.orchestra.run.smithi111.stdout:(27/150): ceph-grafana-dashboards-18.0.0-7531.g 481 kB/s | 28 kB 00:00 2023-12-04T05:36:54.898 INFO:teuthology.orchestra.run.smithi111.stdout:(28/150): ceph-mgr-cephadm-18.0.0-7531.g47c1c54 2.2 MB/s | 133 kB 00:00 2023-12-04T05:36:55.082 INFO:teuthology.orchestra.run.smithi111.stdout:(29/150): ceph-mgr-dashboard-18.0.0-7531.g47c1c 9.6 MB/s | 1.8 MB 00:00 2023-12-04T05:36:55.119 INFO:teuthology.orchestra.run.smithi192.stdout:(97/150): libarrow-doc-9.0.0-9.el9.noarch.rpm 54 kB/s | 24 kB 00:00 2023-12-04T05:36:55.149 INFO:teuthology.orchestra.run.smithi111.stdout:(30/150): librgw2-18.0.0-7531.g47c1c547.el9.x86 3.2 MB/s | 4.8 MB 00:01 2023-12-04T05:36:55.274 INFO:teuthology.orchestra.run.smithi111.stdout:(31/150): ceph-mgr-modules-core-18.0.0-7531.g47 1.9 MB/s | 243 kB 00:00 2023-12-04T05:36:55.352 INFO:teuthology.orchestra.run.smithi192.stdout:(98/150): gperftools-libs-2.9.1-2.el9.x86_64.rp 363 kB/s | 309 kB 00:00 2023-12-04T05:36:55.366 INFO:teuthology.orchestra.run.smithi111.stdout:(32/150): rbd-mirror-18.0.0-7531.g47c1c547.el9. 4.4 MB/s | 3.0 MB 00:00 2023-12-04T05:36:55.369 INFO:teuthology.orchestra.run.smithi192.stdout:(99/150): liboath-2.6.7-2.el9.x86_64.rpm 194 kB/s | 49 kB 00:00 2023-12-04T05:36:55.383 INFO:teuthology.orchestra.run.smithi111.stdout:(33/150): ceph-mgr-rook-18.0.0-7531.g47c1c547.e 463 kB/s | 50 kB 00:00 2023-12-04T05:36:55.425 INFO:teuthology.orchestra.run.smithi111.stdout:(34/150): ceph-prometheus-alerts-18.0.0-7531.g4 259 kB/s | 15 kB 00:00 2023-12-04T05:36:55.494 INFO:teuthology.orchestra.run.smithi192.stdout:(100/150): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2023-12-04T05:36:55.500 INFO:teuthology.orchestra.run.smithi111.stdout:(35/150): ceph-volume-18.0.0-7531.g47c1c547.el9 2.2 MB/s | 262 kB 00:00 2023-12-04T05:36:55.567 INFO:teuthology.orchestra.run.smithi111.stdout:(36/150): cephfs-top-18.0.0-7531.g47c1c547.el9. 275 kB/s | 18 kB 00:00 2023-12-04T05:36:55.619 INFO:teuthology.orchestra.run.smithi192.stdout:(101/150): luarocks-3.9.2-1.el9.noarch.rpm 604 kB/s | 151 kB 00:00 2023-12-04T05:36:55.701 INFO:teuthology.orchestra.run.smithi111.stdout:(37/150): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-12-04T05:36:55.734 INFO:teuthology.orchestra.run.smithi111.stdout:(38/150): cephadm-18.0.0-7531.g47c1c547.el9.noa 2.4 MB/s | 773 kB 00:00 2023-12-04T05:36:55.819 INFO:teuthology.orchestra.run.smithi192.stdout:(102/150): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.5 MB/s | 837 kB 00:00 2023-12-04T05:36:55.859 INFO:teuthology.orchestra.run.smithi111.stdout:(39/150): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2023-12-04T05:36:55.886 INFO:teuthology.orchestra.run.smithi192.stdout:(103/150): libarrow-9.0.0-9.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2023-12-04T05:36:55.901 INFO:teuthology.orchestra.run.smithi111.stdout:(40/150): boost-program-options-1.75.0-8.el9.x8 634 kB/s | 106 kB 00:00 2023-12-04T05:36:55.935 INFO:teuthology.orchestra.run.smithi111.stdout:(41/150): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 920 kB 00:00 2023-12-04T05:36:55.936 INFO:teuthology.orchestra.run.smithi192.stdout:(104/150): python3-asyncssh-2.13.2-3.el9.noarch 1.7 MB/s | 548 kB 00:00 2023-12-04T05:36:55.960 INFO:teuthology.orchestra.run.smithi111.stdout:(42/150): elfutils-libelf-devel-0.190-1.el9.x86 216 kB/s | 21 kB 00:00 2023-12-04T05:36:55.961 INFO:teuthology.orchestra.run.smithi192.stdout:(105/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 306 kB/s | 43 kB 00:00 2023-12-04T05:36:55.986 INFO:teuthology.orchestra.run.smithi111.stdout:(43/150): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2023-12-04T05:36:56.011 INFO:teuthology.orchestra.run.smithi111.stdout:(44/150): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2023-12-04T05:36:56.029 INFO:teuthology.orchestra.run.smithi192.stdout:(106/150): python3-cachetools-4.2.4-1.el9.noarc 226 kB/s | 32 kB 00:00 2023-12-04T05:36:56.054 INFO:teuthology.orchestra.run.smithi192.stdout:(107/150): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2023-12-04T05:36:56.121 INFO:teuthology.orchestra.run.smithi192.stdout:(108/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2023-12-04T05:36:56.155 INFO:teuthology.orchestra.run.smithi192.stdout:(109/150): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2023-12-04T05:36:56.178 INFO:teuthology.orchestra.run.smithi111.stdout:(45/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2023-12-04T05:36:56.204 INFO:teuthology.orchestra.run.smithi111.stdout:(46/150): flexiblas-openblas-openmp-3.0.4-8.el9 79 kB/s | 17 kB 00:00 2023-12-04T05:36:56.214 INFO:teuthology.orchestra.run.smithi192.stdout:(110/150): python3-google-auth-2.23.4-1.el9.noa 1.2 MB/s | 197 kB 00:00 2023-12-04T05:36:56.228 INFO:teuthology.orchestra.run.smithi111.stdout:(47/150): jq-1.6-16.el9.x86_64.rpm 861 kB/s | 186 kB 00:00 2023-12-04T05:36:56.248 INFO:teuthology.orchestra.run.smithi192.stdout:(111/150): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2023-12-04T05:36:56.253 INFO:teuthology.orchestra.run.smithi111.stdout:(48/150): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2023-12-04T05:36:56.279 INFO:teuthology.orchestra.run.smithi111.stdout:(49/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-12-04T05:36:56.289 INFO:teuthology.orchestra.run.smithi192.stdout:(112/150): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2023-12-04T05:36:56.304 INFO:teuthology.orchestra.run.smithi111.stdout:(50/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 589 kB/s | 44 kB 00:00 2023-12-04T05:36:56.338 INFO:teuthology.orchestra.run.smithi111.stdout:(51/150): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2023-12-04T05:36:56.356 INFO:teuthology.orchestra.run.smithi192.stdout:(113/150): python3-jaraco-collections-3.0.0-8.e 163 kB/s | 23 kB 00:00 2023-12-04T05:36:56.363 INFO:teuthology.orchestra.run.smithi111.stdout:(52/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-12-04T05:36:56.381 INFO:teuthology.orchestra.run.smithi192.stdout:(114/150): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2023-12-04T05:36:56.388 INFO:teuthology.orchestra.run.smithi111.stdout:(53/150): libuuid-devel-2.37.4-15.el9.x86_64.rp 296 kB/s | 24 kB 00:00 2023-12-04T05:36:56.413 INFO:teuthology.orchestra.run.smithi111.stdout:(54/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 625 kB/s | 46 kB 00:00 2023-12-04T05:36:56.423 INFO:teuthology.orchestra.run.smithi192.stdout:(115/150): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2023-12-04T05:36:56.439 INFO:teuthology.orchestra.run.smithi111.stdout:(55/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-12-04T05:36:56.472 INFO:teuthology.orchestra.run.smithi111.stdout:(56/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-12-04T05:36:56.498 INFO:teuthology.orchestra.run.smithi111.stdout:(57/150): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2023-12-04T05:36:56.523 INFO:teuthology.orchestra.run.smithi111.stdout:(58/150): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2023-12-04T05:36:56.531 INFO:teuthology.orchestra.run.smithi192.stdout:(116/150): python3-jwt+crypto-2.4.0-1.el9.noarc 52 kB/s | 9.0 kB 00:00 2023-12-04T05:36:56.557 INFO:teuthology.orchestra.run.smithi192.stdout:(117/150): python3-jwt-2.4.0-1.el9.noarch.rpm 233 kB/s | 41 kB 00:00 2023-12-04T05:36:56.607 INFO:teuthology.orchestra.run.smithi192.stdout:(118/150): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2023-12-04T05:36:56.658 INFO:teuthology.orchestra.run.smithi111.stdout:(59/150): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2023-12-04T05:36:56.674 INFO:teuthology.orchestra.run.smithi192.stdout:(119/150): python3-logutils-0.3.5-21.el9.noarch 326 kB/s | 46 kB 00:00 2023-12-04T05:36:56.699 INFO:teuthology.orchestra.run.smithi111.stdout:(60/150): postgresql-13.11-1.el9.x86_64.rpm 7.6 MB/s | 1.5 MB 00:00 2023-12-04T05:36:56.699 INFO:teuthology.orchestra.run.smithi192.stdout:(120/150): python3-more-itertools-8.12.0-2.el9. 555 kB/s | 79 kB 00:00 2023-12-04T05:36:56.724 INFO:teuthology.orchestra.run.smithi192.stdout:(121/150): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2023-12-04T05:36:56.741 INFO:teuthology.orchestra.run.smithi111.stdout:(61/150): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 811 kB 00:00 2023-12-04T05:36:56.766 INFO:teuthology.orchestra.run.smithi111.stdout:(62/150): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2023-12-04T05:36:56.833 INFO:teuthology.orchestra.run.smithi192.stdout:(122/150): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2023-12-04T05:36:56.858 INFO:teuthology.orchestra.run.smithi192.stdout:(123/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 673 kB/s | 90 kB 00:00 2023-12-04T05:36:56.883 INFO:teuthology.orchestra.run.smithi111.stdout:(63/150): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-12-04T05:36:56.883 INFO:teuthology.orchestra.run.smithi192.stdout:(124/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2023-12-04T05:36:56.908 INFO:teuthology.orchestra.run.smithi111.stdout:(64/150): python3-jinja2-2.11.3-4.el9.noarch.rp 1.6 MB/s | 228 kB 00:00 2023-12-04T05:36:56.975 INFO:teuthology.orchestra.run.smithi192.stdout:(125/150): python3-repoze-lru-0.7-16.el9.noarch 216 kB/s | 31 kB 00:00 2023-12-04T05:36:57.001 INFO:teuthology.orchestra.run.smithi192.stdout:(126/150): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2023-12-04T05:36:57.008 INFO:teuthology.orchestra.run.smithi111.stdout:(65/150): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2023-12-04T05:36:57.026 INFO:teuthology.orchestra.run.smithi192.stdout:(127/150): python3-rsa-4.9-2.el9.noarch.rpm 415 kB/s | 59 kB 00:00 2023-12-04T05:36:57.034 INFO:teuthology.orchestra.run.smithi111.stdout:(66/150): python3-jsonpatch-1.21-16.el9.noarch. 162 kB/s | 24 kB 00:00 2023-12-04T05:36:57.059 INFO:teuthology.orchestra.run.smithi111.stdout:(67/150): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2023-12-04T05:36:57.092 INFO:teuthology.orchestra.run.smithi111.stdout:(68/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2023-12-04T05:36:57.117 INFO:teuthology.orchestra.run.smithi111.stdout:(69/150): python3-markupsafe-1.1.1-12.el9.x86_6 545 kB/s | 32 kB 00:00 2023-12-04T05:36:57.118 INFO:teuthology.orchestra.run.smithi192.stdout:(128/150): python3-tempora-5.0.0-2.el9.noarch.r 252 kB/s | 36 kB 00:00 2023-12-04T05:36:57.143 INFO:teuthology.orchestra.run.smithi192.stdout:(129/150): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2023-12-04T05:36:57.193 INFO:teuthology.orchestra.run.smithi111.stdout:(70/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2023-12-04T05:36:57.218 INFO:teuthology.orchestra.run.smithi111.stdout:(71/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2023-12-04T05:36:57.219 INFO:teuthology.orchestra.run.smithi192.stdout:(130/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2023-12-04T05:36:57.243 INFO:teuthology.orchestra.run.smithi111.stdout:(72/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2023-12-04T05:36:57.268 INFO:teuthology.orchestra.run.smithi111.stdout:(73/150): python3-prettytable-0.7.2-27.el9.noar 552 kB/s | 41 kB 00:00 2023-12-04T05:36:57.277 INFO:teuthology.orchestra.run.smithi192.stdout:(131/150): python3-websocket-client-1.2.3-2.el9 565 kB/s | 90 kB 00:00 2023-12-04T05:36:57.294 INFO:teuthology.orchestra.run.smithi111.stdout:(74/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-12-04T05:36:57.311 INFO:teuthology.orchestra.run.smithi192.stdout:(132/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2023-12-04T05:36:57.319 INFO:teuthology.orchestra.run.smithi111.stdout:(75/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2023-12-04T05:36:57.344 INFO:teuthology.orchestra.run.smithi111.stdout:(76/150): python3-pytz-2021.1-5.el9.noarch.rpm 627 kB/s | 47 kB 00:00 2023-12-04T05:36:57.361 INFO:teuthology.orchestra.run.smithi192.stdout:(133/150): python3-zc-lockfile-2.0-10.el9.noarc 140 kB/s | 20 kB 00:00 2023-12-04T05:36:57.386 INFO:teuthology.orchestra.run.smithi111.stdout:(77/150): python3-requests-oauthlib-1.3.0-12.el 643 kB/s | 43 kB 00:00 2023-12-04T05:36:57.411 INFO:teuthology.orchestra.run.smithi192.stdout:(134/150): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-12-04T05:36:57.470 INFO:teuthology.orchestra.run.smithi192.stdout:(135/150): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-12-04T05:36:57.486 INFO:teuthology.orchestra.run.smithi111.stdout:(78/150): python3-toml-0.10.2-6.el9.noarch.rpm 278 kB/s | 39 kB 00:00 2023-12-04T05:36:57.503 INFO:teuthology.orchestra.run.smithi192.stdout:(136/150): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-12-04T05:36:57.537 INFO:teuthology.orchestra.run.smithi192.stdout:(137/150): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-12-04T05:36:57.564 INFO:teuthology.orchestra.run.smithi192.stdout:(138/150): elfutils-debuginfod-client-0.190-1.e 1.4 MB/s | 36 kB 00:00 2023-12-04T05:36:57.604 INFO:teuthology.orchestra.run.smithi192.stdout:(139/150): elfutils-libelf-0.190-1.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2023-12-04T05:36:57.646 INFO:teuthology.orchestra.run.smithi192.stdout:(140/150): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-12-04T05:36:57.662 INFO:teuthology.orchestra.run.smithi111.stdout:(79/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2023-12-04T05:36:57.680 INFO:teuthology.orchestra.run.smithi192.stdout:(141/150): libacl-2.3.1-4.el9.x86_64.rpm 660 kB/s | 22 kB 00:00 2023-12-04T05:36:57.695 INFO:teuthology.orchestra.run.smithi111.stdout:(80/150): socat-1.7.4.1-5.el9.x86_64.rpm 980 kB/s | 300 kB 00:00 2023-12-04T05:36:57.720 INFO:teuthology.orchestra.run.smithi111.stdout:(81/150): uuid-1.6.2-55.el9.x86_64.rpm 244 kB/s | 56 kB 00:00 2023-12-04T05:36:57.730 INFO:teuthology.orchestra.run.smithi192.stdout:(142/150): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-12-04T05:36:57.781 INFO:teuthology.orchestra.run.smithi192.stdout:(143/150): librbd1-18.0.0-7531.g47c1c547.el9.x8 8.2 MB/s | 3.0 MB 00:00 2023-12-04T05:36:57.795 INFO:teuthology.orchestra.run.smithi111.stdout:(82/150): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 648 kB/s | 85 kB 00:00 2023-12-04T05:36:57.871 INFO:teuthology.orchestra.run.smithi111.stdout:(83/150): zlib-devel-1.2.11-40.el9.x86_64.rpm 294 kB/s | 44 kB 00:00 2023-12-04T05:36:57.896 INFO:teuthology.orchestra.run.smithi111.stdout:(84/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 357 kB/s | 63 kB 00:00 2023-12-04T05:36:57.930 INFO:teuthology.orchestra.run.smithi111.stdout:(85/150): fuse-2.9.9-15.el9.x86_64.rpm 583 kB/s | 79 kB 00:00 2023-12-04T05:36:57.955 INFO:teuthology.orchestra.run.smithi111.stdout:(86/150): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-12-04T05:36:57.989 INFO:teuthology.orchestra.run.smithi111.stdout:(87/150): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-12-04T05:36:58.015 INFO:teuthology.orchestra.run.smithi111.stdout:(88/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2023-12-04T05:36:58.058 INFO:teuthology.orchestra.run.smithi111.stdout:(89/150): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2023-12-04T05:36:58.082 INFO:teuthology.orchestra.run.smithi111.stdout:(90/150): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-12-04T05:36:58.107 INFO:teuthology.orchestra.run.smithi111.stdout:(91/150): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2023-12-04T05:36:58.132 INFO:teuthology.orchestra.run.smithi111.stdout:(92/150): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-12-04T05:36:58.158 INFO:teuthology.orchestra.run.smithi111.stdout:(93/150): python3-urllib3-1.26.5-4.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2023-12-04T05:36:58.230 INFO:teuthology.orchestra.run.smithi192.stdout:(144/150): librados2-18.0.0-7531.g47c1c547.el9. 3.8 MB/s | 3.3 MB 00:00 2023-12-04T05:36:58.391 INFO:teuthology.orchestra.run.smithi111.stdout:(94/150): python3-devel-3.9.17-2.el9.x86_64.rpm 950 kB/s | 245 kB 00:00 2023-12-04T05:36:58.558 INFO:teuthology.orchestra.run.smithi111.stdout:(95/150): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.2 MB/s | 309 kB 00:00 2023-12-04T05:36:58.692 INFO:teuthology.orchestra.run.smithi111.stdout:(96/150): libarrow-9.0.0-9.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2023-12-04T05:36:58.717 INFO:teuthology.orchestra.run.smithi111.stdout:(97/150): libarrow-doc-9.0.0-9.el9.noarch.rpm 984 kB/s | 24 kB 00:00 2023-12-04T05:36:58.742 INFO:teuthology.orchestra.run.smithi111.stdout:(98/150): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2023-12-04T05:36:58.759 INFO:teuthology.orchestra.run.smithi111.stdout:(99/150): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2023-12-04T05:36:58.784 INFO:teuthology.orchestra.run.smithi111.stdout:(100/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2023-12-04T05:36:58.810 INFO:teuthology.orchestra.run.smithi111.stdout:(101/150): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2023-12-04T05:36:58.901 INFO:teuthology.orchestra.run.smithi111.stdout:(102/150): parquet-libs-9.0.0-9.el9.x86_64.rpm 7.0 MB/s | 837 kB 00:00 2023-12-04T05:36:58.935 INFO:teuthology.orchestra.run.smithi111.stdout:(103/150): python3-asyncssh-2.13.2-3.el9.noarch 4.3 MB/s | 548 kB 00:00 2023-12-04T05:36:58.960 INFO:teuthology.orchestra.run.smithi111.stdout:(104/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 751 kB/s | 43 kB 00:00 2023-12-04T05:36:58.986 INFO:teuthology.orchestra.run.smithi111.stdout:(105/150): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2023-12-04T05:36:59.013 INFO:teuthology.orchestra.run.smithi111.stdout:(106/150): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2023-12-04T05:36:59.071 INFO:teuthology.orchestra.run.smithi111.stdout:(107/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-12-04T05:36:59.103 INFO:teuthology.orchestra.run.smithi111.stdout:(108/150): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2023-12-04T05:36:59.128 INFO:teuthology.orchestra.run.smithi111.stdout:(109/150): python3-google-auth-2.23.4-1.el9.noa 3.3 MB/s | 197 kB 00:00 2023-12-04T05:36:59.154 INFO:teuthology.orchestra.run.smithi111.stdout:(110/150): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2023-12-04T05:36:59.238 INFO:teuthology.orchestra.run.smithi111.stdout:(111/150): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2023-12-04T05:36:59.264 INFO:teuthology.orchestra.run.smithi111.stdout:(112/150): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2023-12-04T05:36:59.296 INFO:teuthology.orchestra.run.smithi111.stdout:(113/150): python3-jaraco-text-3.2.0-6.el9.noar 592 kB/s | 20 kB 00:00 2023-12-04T05:36:59.330 INFO:teuthology.orchestra.run.smithi111.stdout:(114/150): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2023-12-04T05:36:59.356 INFO:teuthology.orchestra.run.smithi111.stdout:(115/150): python3-jwt+crypto-2.4.0-1.el9.noarc 154 kB/s | 9.0 kB 00:00 2023-12-04T05:36:59.414 INFO:teuthology.orchestra.run.smithi111.stdout:(116/150): openssl-devel-3.0.7-25.el9.x86_64.rp 3.2 MB/s | 4.1 MB 00:01 2023-12-04T05:36:59.441 INFO:teuthology.orchestra.run.smithi111.stdout:(117/150): python3-jwt-2.4.0-1.el9.noarch.rpm 374 kB/s | 41 kB 00:00 2023-12-04T05:36:59.475 INFO:teuthology.orchestra.run.smithi111.stdout:(118/150): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2023-12-04T05:36:59.498 INFO:teuthology.orchestra.run.smithi111.stdout:(119/150): python3-logutils-0.3.5-21.el9.noarch 563 kB/s | 46 kB 00:00 2023-12-04T05:36:59.548 INFO:teuthology.orchestra.run.smithi111.stdout:(120/150): python3-natsort-7.1.1-5.el9.noarch.r 772 kB/s | 58 kB 00:00 2023-12-04T05:36:59.599 INFO:teuthology.orchestra.run.smithi111.stdout:(121/150): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2023-12-04T05:36:59.624 INFO:teuthology.orchestra.run.smithi111.stdout:(122/150): python3-more-itertools-8.12.0-2.el9. 523 kB/s | 79 kB 00:00 2023-12-04T05:36:59.649 INFO:teuthology.orchestra.run.smithi111.stdout:(123/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2023-12-04T05:36:59.675 INFO:teuthology.orchestra.run.smithi111.stdout:(124/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2023-12-04T05:36:59.700 INFO:teuthology.orchestra.run.smithi111.stdout:(125/150): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2023-12-04T05:36:59.725 INFO:teuthology.orchestra.run.smithi111.stdout:(126/150): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2023-12-04T05:36:59.751 INFO:teuthology.orchestra.run.smithi111.stdout:(127/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2023-12-04T05:36:59.826 INFO:teuthology.orchestra.run.smithi111.stdout:(128/150): python3-typing-extensions-4.4.0-2.el 520 kB/s | 51 kB 00:00 2023-12-04T05:36:59.860 INFO:teuthology.orchestra.run.smithi111.stdout:(129/150): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2023-12-04T05:36:59.885 INFO:teuthology.orchestra.run.smithi111.stdout:(130/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2023-12-04T05:36:59.919 INFO:teuthology.orchestra.run.smithi111.stdout:(131/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2023-12-04T05:36:59.945 INFO:teuthology.orchestra.run.smithi111.stdout:(132/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2023-12-04T05:36:59.977 INFO:teuthology.orchestra.run.smithi111.stdout:(133/150): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2023-12-04T05:36:59.994 INFO:teuthology.orchestra.run.smithi111.stdout:(134/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2023-12-04T05:37:00.036 INFO:teuthology.orchestra.run.smithi111.stdout:(135/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2023-12-04T05:37:00.078 INFO:teuthology.orchestra.run.smithi111.stdout:(136/150): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-12-04T05:37:00.112 INFO:teuthology.orchestra.run.smithi111.stdout:(137/150): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-12-04T05:37:00.146 INFO:teuthology.orchestra.run.smithi111.stdout:(138/150): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-12-04T05:37:00.188 INFO:teuthology.orchestra.run.smithi111.stdout:(139/150): elfutils-libelf-0.190-1.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2023-12-04T05:37:00.237 INFO:teuthology.orchestra.run.smithi111.stdout:(140/150): elfutils-libs-0.190-1.el9.x86_64.rpm 5.0 MB/s | 255 kB 00:00 2023-12-04T05:37:00.271 INFO:teuthology.orchestra.run.smithi111.stdout:(141/150): libacl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 22 kB 00:00 2023-12-04T05:37:00.380 INFO:teuthology.orchestra.run.smithi111.stdout:(142/150): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2023-12-04T05:37:00.522 INFO:teuthology.orchestra.run.smithi111.stdout:(143/150): librbd1-18.0.0-7531.g47c1c547.el9.x8 5.7 MB/s | 3.0 MB 00:00 2023-12-04T05:37:00.974 INFO:teuthology.orchestra.run.smithi111.stdout:(144/150): librados2-18.0.0-7531.g47c1c547.el9. 3.3 MB/s | 3.3 MB 00:01 2023-12-04T05:37:01.498 INFO:teuthology.orchestra.run.smithi111.stdout:(145/150): NetworkManager-1.45.7-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2023-12-04T05:37:01.631 INFO:teuthology.orchestra.run.smithi111.stdout:(146/150): NetworkManager-team-1.45.7-1.el9.x86 66 kB/s | 43 kB 00:00 2023-12-04T05:37:01.673 INFO:teuthology.orchestra.run.smithi111.stdout:(147/150): NetworkManager-libnm-1.45.7-1.el9.x8 1.5 MB/s | 1.8 MB 00:01 2023-12-04T05:37:02.357 INFO:teuthology.orchestra.run.smithi111.stdout:(148/150): NetworkManager-tui-1.45.7-1.el9.x86_ 287 kB/s | 246 kB 00:00 2023-12-04T05:37:02.674 INFO:teuthology.orchestra.run.smithi111.stdout:(149/150): openssl-3.0.7-25.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:01 2023-12-04T05:37:02.751 INFO:teuthology.orchestra.run.smithi111.stdout:(150/150): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2023-12-04T05:37:02.755 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-12-04T05:37:02.755 INFO:teuthology.orchestra.run.smithi111.stdout:Total 12 MB/s | 205 MB 00:16 2023-12-04T05:37:02.933 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-12-04T05:37:03.117 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-12-04T05:37:03.117 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-12-04T05:37:04.930 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-12-04T05:37:04.930 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-12-04T05:37:09.951 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-1.45.7-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-1.45.7-1.el9.x86_64.rpm [Failed to connect to mirror.facebook.net port 80: Connection refused] 2023-12-04T05:37:09.951 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-libnm-1.45.7-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-libnm-1.45.7-1.el9.x86_64.rpm [Failed to connect to mirror.facebook.net port 80: Connection refused] 2023-12-04T05:37:10.311 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-team-1.45.7-1.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-team-1.45.7-1.el9.x86_64.rpm [Failed to connect to mirror.facebook.net port 80: Connection refused] 2023-12-04T05:37:10.905 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-12-04T05:37:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/165 2023-12-04T05:37:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/165 2023-12-04T05:37:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/165 2023-12-04T05:37:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/165 2023-12-04T05:37:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2023-12-04T05:37:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2023-12-04T05:37:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2023-12-04T05:37:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 8/165 2023-12-04T05:37:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 8/165 2023-12-04T05:37:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 9/165 2023-12-04T05:37:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 9/165 2023-12-04T05:37:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:18.0.0-7531.g47c1c547.el9.x86_64 10/165 2023-12-04T05:37:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/165 2023-12-04T05:37:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/165 2023-12-04T05:37:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/165 2023-12-04T05:37:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 14/165 2023-12-04T05:37:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 14/165 2023-12-04T05:37:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86 15/165 2023-12-04T05:37:12.525 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86 15/165 2023-12-04T05:37:12.651 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-04T05:37:12.695 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-04T05:37:12.790 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-04T05:37:12.846 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-04T05:37:13.063 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-04T05:37:13.169 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/165 2023-12-04T05:37:13.210 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/165 2023-12-04T05:37:13.276 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/165 2023-12-04T05:37:13.331 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/165 2023-12-04T05:37:13.363 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/165 2023-12-04T05:37:13.398 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/165 2023-12-04T05:37:13.440 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/165 2023-12-04T05:37:13.509 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/165 2023-12-04T05:37:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/165 2023-12-04T05:37:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/165 2023-12-04T05:37:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2023-12-04T05:37:13.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2023-12-04T05:37:13.736 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:18.0.0-7531.g47c1c547.el9. 30/165 2023-12-04T05:37:13.779 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:18.0.0-7531.g47c1c547.el 31/165 2023-12-04T05:37:13.808 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:18.0.0-7531.g47c1c547.el9.x86_6 32/165 2023-12-04T05:37:13.876 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/165 2023-12-04T05:37:13.962 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/165 2023-12-04T05:37:14.104 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/165 2023-12-04T05:37:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/165 2023-12-04T05:37:14.294 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-1.45.7-1.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-1.45.7-1.el9.x86_64.rpm [OpenSSL SSL_read: Connection reset by peer, errno 104] 2023-12-04T05:37:14.334 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-libnm-1.45.7-1.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-libnm-1.45.7-1.el9.x86_64.rpm [OpenSSL SSL_read: Connection reset by peer, errno 104] 2023-12-04T05:37:14.596 INFO:teuthology.orchestra.run.smithi192.stdout:(145/150): NetworkManager-libnm-1.45.7-1.el9.x8 108 kB/s | 1.8 MB 00:16 2023-12-04T05:37:14.638 INFO:teuthology.orchestra.run.smithi192.stdout:(146/150): NetworkManager-1.45.7-1.el9.x86_64.r 137 kB/s | 2.3 MB 00:16 2023-12-04T05:37:14.655 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-team-1.45.7-1.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-team-1.45.7-1.el9.x86_64.rpm [OpenSSL SSL_read: Connection reset by peer, errno 104] 2023-12-04T05:37:14.675 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/165 2023-12-04T05:37:14.752 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/165 2023-12-04T05:37:14.771 INFO:teuthology.orchestra.run.smithi192.stdout:(147/150): NetworkManager-team-1.45.7-1.el9.x86 2.6 kB/s | 43 kB 00:16 2023-12-04T05:37:14.809 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 39/165 2023-12-04T05:37:14.850 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 40/165 2023-12-04T05:37:14.888 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:18.0.0-7531.g47c1c547.el9.x86_6 41/165 2023-12-04T05:37:14.930 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/165 2023-12-04T05:37:15.065 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 43/165 2023-12-04T05:37:15.114 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 44/165 2023-12-04T05:37:15.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-3.el9.x86_64 45/165 2023-12-04T05:37:15.258 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 46/165 2023-12-04T05:37:15.313 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/165 2023-12-04T05:37:15.385 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/165 2023-12-04T05:37:15.417 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/165 2023-12-04T05:37:15.454 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/165 2023-12-04T05:37:15.500 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/165 2023-12-04T05:37:15.547 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/165 2023-12-04T05:37:15.587 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 53/165 2023-12-04T05:37:15.737 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 54/165 2023-12-04T05:37:15.790 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 55/165 2023-12-04T05:37:15.831 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 56/165 2023-12-04T05:37:15.925 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 57/165 2023-12-04T05:37:15.958 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 58/165 2023-12-04T05:37:15.999 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 59/165 2023-12-04T05:37:16.030 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 60/165 2023-12-04T05:37:16.124 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 61/165 2023-12-04T05:37:16.175 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/165 2023-12-04T05:37:16.417 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/165 2023-12-04T05:37:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/165 2023-12-04T05:37:16.663 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/165 2023-12-04T05:37:16.712 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/165 2023-12-04T05:37:16.851 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/165 2023-12-04T05:37:16.898 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/165 2023-12-04T05:37:16.936 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/165 2023-12-04T05:37:16.969 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/165 2023-12-04T05:37:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/165 2023-12-04T05:37:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-15.el9.x86_64 72/165 2023-12-04T05:37:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/165 2023-12-04T05:37:17.093 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/165 2023-12-04T05:37:17.131 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/165 2023-12-04T05:37:17.162 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/165 2023-12-04T05:37:17.237 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/165 2023-12-04T05:37:17.272 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 78/165 2023-12-04T05:37:17.306 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/165 2023-12-04T05:37:17.347 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/165 2023-12-04T05:37:17.382 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/165 2023-12-04T05:37:17.470 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/165 2023-12-04T05:37:17.501 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 83/165 2023-12-04T05:37:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/165 2023-12-04T05:37:17.643 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/165 2023-12-04T05:37:18.371 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/165 2023-12-04T05:37:18.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/165 2023-12-04T05:37:18.449 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/165 2023-12-04T05:37:18.481 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/165 2023-12-04T05:37:18.642 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.21-2.el9.x86_64 90/165 2023-12-04T05:37:18.663 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 91/165 2023-12-04T05:37:18.732 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 92/165 2023-12-04T05:37:18.954 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] NetworkManager-tui-1.45.7-1.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-tui-1.45.7-1.el9.x86_64.rpm [OpenSSL SSL_read: Connection reset by peer, errno 104] 2023-12-04T05:37:18.980 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] openssl-3.0.7-25.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/openssl-3.0.7-25.el9.x86_64.rpm [OpenSSL SSL_read: Connection reset by peer, errno 104] 2023-12-04T05:37:19.113 INFO:teuthology.orchestra.run.smithi192.stdout:[MIRROR] openssl-libs-3.0.7-25.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/openssl-libs-3.0.7-25.el9.x86_64.rpm [OpenSSL SSL_read: Connection reset by peer, errno 104] 2023-12-04T05:37:19.304 INFO:teuthology.orchestra.run.smithi192.stdout:(148/150): NetworkManager-tui-1.45.7-1.el9.x86_ 52 kB/s | 246 kB 00:04 2023-12-04T05:37:19.382 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 93/165 2023-12-04T05:37:19.493 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 94/165 2023-12-04T05:37:19.504 INFO:teuthology.orchestra.run.smithi192.stdout:(149/150): openssl-3.0.7-25.el9.x86_64.rpm 255 kB/s | 1.2 MB 00:04 2023-12-04T05:37:19.572 INFO:teuthology.orchestra.run.smithi192.stdout:(150/150): openssl-libs-3.0.7-25.el9.x86_64.rpm 460 kB/s | 2.2 MB 00:04 2023-12-04T05:37:19.576 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-12-04T05:37:19.576 INFO:teuthology.orchestra.run.smithi192.stdout:Total 5.9 MB/s | 205 MB 00:34 2023-12-04T05:37:19.749 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-12-04T05:37:19.938 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-12-04T05:37:19.938 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-12-04T05:37:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 95/165 2023-12-04T05:37:20.788 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 96/165 2023-12-04T05:37:20.820 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 97/165 2023-12-04T05:37:20.850 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : jq-1.6-16.el9.x86_64 98/165 2023-12-04T05:37:20.879 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2023-12-04T05:37:20.912 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 100/165 2023-12-04T05:37:21.214 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2023-12-04T05:37:21.242 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 102/165 2023-12-04T05:37:21.332 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 102/165 2023-12-04T05:37:21.372 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:18.0.0-7531.g47c1c547.el9.x86_64 103/165 2023-12-04T05:37:21.407 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2023-12-04T05:37:21.829 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 105/165 2023-12-04T05:37:21.833 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-12-04T05:37:21.834 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-12-04T05:37:21.862 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 106/165 2023-12-04T05:37:21.915 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 106/165 2023-12-04T05:37:21.938 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:18.0.0-7531.g47c1c547.el9.x86_64 107/165 2023-12-04T05:37:24.012 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 108/165 2023-12-04T05:37:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 108/165 2023-12-04T05:37:24.608 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 108/165 2023-12-04T05:37:24.654 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 109/165 2023-12-04T05:37:24.739 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 109/165 2023-12-04T05:37:24.739 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-04T05:37:24.740 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-04T05:37:24.740 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:24.781 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 110/165 2023-12-04T05:37:27.754 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-12-04T05:37:27.883 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/165 2023-12-04T05:37:27.909 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/165 2023-12-04T05:37:27.950 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/165 2023-12-04T05:37:27.992 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/165 2023-12-04T05:37:28.017 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2023-12-04T05:37:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2023-12-04T05:37:28.404 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2023-12-04T05:37:28.460 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 8/165 2023-12-04T05:37:28.579 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 8/165 2023-12-04T05:37:28.603 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 9/165 2023-12-04T05:37:28.694 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 9/165 2023-12-04T05:37:28.730 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:18.0.0-7531.g47c1c547.el9.x86_64 10/165 2023-12-04T05:37:28.768 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/165 2023-12-04T05:37:28.960 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/165 2023-12-04T05:37:29.014 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/165 2023-12-04T05:37:29.041 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 14/165 2023-12-04T05:37:29.151 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 14/165 2023-12-04T05:37:29.175 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86 15/165 2023-12-04T05:37:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86 15/165 2023-12-04T05:37:29.504 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-04T05:37:29.565 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-04T05:37:29.660 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-04T05:37:29.726 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-04T05:37:29.948 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-04T05:37:30.041 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/165 2023-12-04T05:37:30.078 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/165 2023-12-04T05:37:30.175 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/165 2023-12-04T05:37:30.227 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/165 2023-12-04T05:37:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/165 2023-12-04T05:37:30.309 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/165 2023-12-04T05:37:30.349 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/165 2023-12-04T05:37:30.421 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/165 2023-12-04T05:37:30.457 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/165 2023-12-04T05:37:30.497 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/165 2023-12-04T05:37:30.531 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2023-12-04T05:37:30.619 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2023-12-04T05:37:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:18.0.0-7531.g47c1c547.el9. 30/165 2023-12-04T05:37:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:18.0.0-7531.g47c1c547.el 31/165 2023-12-04T05:37:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:18.0.0-7531.g47c1c547.el9.x86_6 32/165 2023-12-04T05:37:30.788 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/165 2023-12-04T05:37:30.874 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/165 2023-12-04T05:37:31.007 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/165 2023-12-04T05:37:31.062 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/165 2023-12-04T05:37:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/165 2023-12-04T05:37:31.664 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/165 2023-12-04T05:37:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 39/165 2023-12-04T05:37:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 40/165 2023-12-04T05:37:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:18.0.0-7531.g47c1c547.el9.x86_6 41/165 2023-12-04T05:37:31.833 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/165 2023-12-04T05:37:31.976 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 43/165 2023-12-04T05:37:32.025 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 44/165 2023-12-04T05:37:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 45/165 2023-12-04T05:37:32.170 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 46/165 2023-12-04T05:37:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/165 2023-12-04T05:37:32.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/165 2023-12-04T05:37:32.329 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/165 2023-12-04T05:37:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/165 2023-12-04T05:37:32.412 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/165 2023-12-04T05:37:32.457 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/165 2023-12-04T05:37:32.502 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 53/165 2023-12-04T05:37:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 54/165 2023-12-04T05:37:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 55/165 2023-12-04T05:37:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 56/165 2023-12-04T05:37:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 57/165 2023-12-04T05:37:32.843 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 58/165 2023-12-04T05:37:32.884 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 59/165 2023-12-04T05:37:32.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 60/165 2023-12-04T05:37:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 61/165 2023-12-04T05:37:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/165 2023-12-04T05:37:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/165 2023-12-04T05:37:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/165 2023-12-04T05:37:33.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/165 2023-12-04T05:37:33.423 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/165 2023-12-04T05:37:33.559 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/165 2023-12-04T05:37:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/165 2023-12-04T05:37:33.630 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/165 2023-12-04T05:37:33.663 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/165 2023-12-04T05:37:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/165 2023-12-04T05:37:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 72/165 2023-12-04T05:37:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/165 2023-12-04T05:37:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/165 2023-12-04T05:37:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/165 2023-12-04T05:37:33.864 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/165 2023-12-04T05:37:33.940 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/165 2023-12-04T05:37:33.974 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 78/165 2023-12-04T05:37:34.015 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/165 2023-12-04T05:37:34.057 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/165 2023-12-04T05:37:34.108 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/165 2023-12-04T05:37:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/165 2023-12-04T05:37:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 83/165 2023-12-04T05:37:34.354 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/165 2023-12-04T05:37:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/165 2023-12-04T05:37:35.134 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/165 2023-12-04T05:37:35.184 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/165 2023-12-04T05:37:35.217 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/165 2023-12-04T05:37:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/165 2023-12-04T05:37:35.429 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 90/165 2023-12-04T05:37:35.464 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 91/165 2023-12-04T05:37:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 92/165 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 110/165 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2023-12-04T05:37:36.096 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2023-12-04T05:37:36.097 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2023-12-04T05:37:36.097 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:36.202 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 93/165 2023-12-04T05:37:36.259 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 111/165 2023-12-04T05:37:36.275 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 111/165 2023-12-04T05:37:36.275 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:36.276 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-04T05:37:36.277 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:36.311 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 94/165 2023-12-04T05:37:37.549 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 95/165 2023-12-04T05:37:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 96/165 2023-12-04T05:37:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 97/165 2023-12-04T05:37:37.693 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-16.el9.x86_64 98/165 2023-12-04T05:37:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2023-12-04T05:37:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 100/165 2023-12-04T05:37:37.846 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 112/165 2023-12-04T05:37:37.871 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 112/165 2023-12-04T05:37:37.871 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:37.871 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-04T05:37:37.871 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-04T05:37:37.871 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-04T05:37:37.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2023-12-04T05:37:38.071 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 113/165 2023-12-04T05:37:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 102/165 2023-12-04T05:37:38.094 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 113/165 2023-12-04T05:37:38.094 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:38.094 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-04T05:37:38.094 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-04T05:37:38.094 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-04T05:37:38.094 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:38.184 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 102/165 2023-12-04T05:37:38.214 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:18.0.0-7531.g47c1c547.el9.x86_64 103/165 2023-12-04T05:37:38.250 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2023-12-04T05:37:38.547 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 114/165 2023-12-04T05:37:38.569 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 114/165 2023-12-04T05:37:38.569 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:38.569 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-04T05:37:38.570 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-04T05:37:38.570 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-04T05:37:38.570 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:38.665 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2023-12-04T05:37:38.681 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 105/165 2023-12-04T05:37:38.712 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 106/165 2023-12-04T05:37:38.783 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 106/165 2023-12-04T05:37:38.806 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:18.0.0-7531.g47c1c547.el9.x86_64 107/165 2023-12-04T05:37:39.118 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 116/165 2023-12-04T05:37:39.139 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 116/165 2023-12-04T05:37:39.169 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7531.g47c1c547.e 117/165 2023-12-04T05:37:39.230 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7531.g47c1c547. 118/165 2023-12-04T05:37:39.253 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noa 119/165 2023-12-04T05:37:39.477 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noa 119/165 2023-12-04T05:37:39.504 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.n 120/165 2023-12-04T05:37:40.470 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.n 120/165 2023-12-04T05:37:40.496 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c 121/165 2023-12-04T05:37:40.595 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c 121/165 2023-12-04T05:37:40.753 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7531.g47c1c547.el 122/165 2023-12-04T05:37:40.782 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 123/165 2023-12-04T05:37:40.804 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 123/165 2023-12-04T05:37:40.804 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:40.804 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-04T05:37:40.805 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-04T05:37:40.805 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-04T05:37:40.805 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:40.849 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 124/165 2023-12-04T05:37:40.859 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 124/165 2023-12-04T05:37:40.895 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 108/165 2023-12-04T05:37:40.914 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:18.0.0-7531.g47c1c547.el9.x86_64 125/165 2023-12-04T05:37:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 108/165 2023-12-04T05:37:41.481 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 108/165 2023-12-04T05:37:41.514 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 109/165 2023-12-04T05:37:41.599 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 109/165 2023-12-04T05:37:41.600 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-04T05:37:41.600 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-04T05:37:41.600 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:41.632 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 110/165 2023-12-04T05:37:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-45.el9.x86_64 126/165 2023-12-04T05:37:41.726 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 127/165 2023-12-04T05:37:41.748 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 127/165 2023-12-04T05:37:41.748 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:41.748 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-04T05:37:41.748 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-04T05:37:41.748 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-04T05:37:41.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:41.785 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7531.g47c1c 128/165 2023-12-04T05:37:41.806 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7531.g47c1c 128/165 2023-12-04T05:37:41.806 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:41.806 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-04T05:37:41.806 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:41.852 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 129/165 2023-12-04T05:37:41.874 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 129/165 2023-12-04T05:37:41.874 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:41.874 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-04T05:37:41.874 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-04T05:37:41.874 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-04T05:37:41.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:42.169 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 130/165 2023-12-04T05:37:42.192 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 130/165 2023-12-04T05:37:42.192 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:42.192 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-04T05:37:42.192 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-04T05:37:42.192 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-04T05:37:42.192 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:46.048 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:18.0.0-7531.g47c1c547.el9.x86_64 131/165 2023-12-04T05:37:46.097 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 132/165 2023-12-04T05:37:46.138 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:18.0.0-7531.g47c1c547.el9.x86_64 133/165 2023-12-04T05:37:46.170 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 134/165 2023-12-04T05:37:46.350 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 135/165 2023-12-04T05:37:46.469 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 136/165 2023-12-04T05:37:46.501 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 137/165 2023-12-04T05:37:46.534 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:18.0.0-7531.g47c1c547.el9.x86_ 138/165 2023-12-04T05:37:46.617 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 139/165 2023-12-04T05:37:46.675 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-04T05:37:46.711 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 141/165 2023-12-04T05:37:46.751 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 142/165 2023-12-04T05:37:46.803 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 143/165 2023-12-04T05:37:47.388 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-04T05:37:47.435 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-04T05:37:47.456 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-04T05:37:48.264 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:18.0.0-7531.g47c1c547.el9.noarch 145/165 2023-12-04T05:37:48.372 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 146/165 2023-12-04T05:37:48.431 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 147/165 2023-12-04T05:37:48.473 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/165 2023-12-04T05:37:48.572 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/165 2023-12-04T05:37:48.602 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/165 2023-12-04T05:37:48.627 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/165 2023-12-04T05:37:48.660 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 152/165 2023-12-04T05:37:48.687 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 153/165 2023-12-04T05:37:48.687 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-04T05:37:48.763 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-04T05:37:48.792 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 155/165 2023-12-04T05:37:48.826 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 156/165 2023-12-04T05:37:48.855 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 157/165 2023-12-04T05:37:48.855 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-04T05:37:48.940 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-04T05:37:48.965 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-04T05:37:48.972 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-04T05:37:48.972 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-04T05:37:49.159 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-04T05:37:49.159 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-04T05:37:49.251 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-04T05:37:49.275 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 162/165 2023-12-04T05:37:49.301 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 163/165 2023-12-04T05:37:49.326 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 164/165 2023-12-04T05:37:49.394 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-04T05:37:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 110/165 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2023-12-04T05:37:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 111/165 2023-12-04T05:37:53.486 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 111/165 2023-12-04T05:37:53.486 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:53.487 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-04T05:37:53.487 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:54.133 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:18.0.0-7531.g47c1c547.el9.x86_64 1/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 2/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 3/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 4/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7531.g47c1c 5/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 6/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 7/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 8/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 9/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 10/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 11/165 2023-12-04T05:37:54.134 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:18.0.0-7531.g47c1c547.el9.x86_64 12/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 13/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:18.0.0-7531.g47c1c547.el9.x86_ 14/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 15/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 16/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:18.0.0-7531.g47c1c547.el9.x86_6 17/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86 18/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 19/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7531.g47c1c547.el 20/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:18.0.0-7531.g47c1c547.el9. 21/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:18.0.0-7531.g47c1c547.el9.x86_6 22/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:18.0.0-7531.g47c1c547.el9.x86_64 23/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:18.0.0-7531.g47c1c547.el9.x86_64 24/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:18.0.0-7531.g47c1c547.el9.x86_64 25/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 26/165 2023-12-04T05:37:54.135 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 27/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:18.0.0-7531.g47c1c547.el9.x86_64 28/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7531.g47c1c547. 29/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noa 30/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.n 31/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c 32/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7531.g47c1c547.el 33/165 2023-12-04T05:37:54.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 34/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7531.g47c1c547.e 35/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 36/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 37/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:18.0.0-7531.g47c1c547.el9.noarch 38/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : jq-1.6-16.el9.x86_64 47/165 2023-12-04T05:37:54.149 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/165 2023-12-04T05:37:54.150 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/165 2023-12-04T05:37:54.151 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/165 2023-12-04T05:37:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 94/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 98/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 99/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/165 2023-12-04T05:37:54.153 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 103/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 110/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/165 2023-12-04T05:37:54.154 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/165 2023-12-04T05:37:54.155 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 136/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 138/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : acl-2.3.1-4.el9.x86_64 142/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : acl-2.3.1-3.el9.x86_64 143/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 144/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/165 2023-12-04T05:37:54.156 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 146/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 147/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 148/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 149/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 154/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 156/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 158/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-04T05:37:54.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 160/165 2023-12-04T05:37:54.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/165 2023-12-04T05:37:54.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/165 2023-12-04T05:37:54.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/165 2023-12-04T05:37:54.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/165 2023-12-04T05:37:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 112/165 2023-12-04T05:37:55.090 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 112/165 2023-12-04T05:37:55.091 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:55.091 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-04T05:37:55.091 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-04T05:37:55.091 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-04T05:37:55.091 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:55.291 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 113/165 2023-12-04T05:37:55.313 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 113/165 2023-12-04T05:37:55.314 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:55.314 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-04T05:37:55.314 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-04T05:37:55.314 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-04T05:37:55.314 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:55.758 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 114/165 2023-12-04T05:37:55.781 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 114/165 2023-12-04T05:37:55.781 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:55.781 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-04T05:37:55.781 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-04T05:37:55.781 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-04T05:37:55.781 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:55.894 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2023-12-04T05:37:56.262 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 116/165 2023-12-04T05:37:56.291 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 116/165 2023-12-04T05:37:56.321 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7531.g47c1c547.e 117/165 2023-12-04T05:37:56.382 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7531.g47c1c547. 118/165 2023-12-04T05:37:56.415 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noa 119/165 2023-12-04T05:37:56.638 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noa 119/165 2023-12-04T05:37:56.674 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.n 120/165 2023-12-04T05:37:56.793 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/165 2023-12-04T05:37:56.793 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:56.793 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2023-12-04T05:37:56.793 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-04T05:37:56.793 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-04T05:37:56.793 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: acl-2.3.1-4.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-04T05:37:56.794 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2023-12-04T05:37:56.796 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: jq-1.6-16.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-04T05:37:56.797 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-04T05:37:56.798 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.11-1.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-04T05:37:56.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-04T05:37:56.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-04T05:37:56.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-04T05:37:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-04T05:37:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-3.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-04T05:37:56.804 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-04T05:37:56.805 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-04T05:37:56.805 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:37:56.805 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-12-04T05:37:57.365 DEBUG:teuthology.parallel:result is None 2023-12-04T05:37:57.644 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.n 120/165 2023-12-04T05:37:57.674 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c 121/165 2023-12-04T05:37:57.772 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c 121/165 2023-12-04T05:37:57.926 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7531.g47c1c547.el 122/165 2023-12-04T05:37:57.984 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 123/165 2023-12-04T05:37:58.007 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 123/165 2023-12-04T05:37:58.007 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:58.007 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-04T05:37:58.008 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-04T05:37:58.008 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-04T05:37:58.008 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:58.052 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 124/165 2023-12-04T05:37:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 124/165 2023-12-04T05:37:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:18.0.0-7531.g47c1c547.el9.x86_64 125/165 2023-12-04T05:37:58.894 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 126/165 2023-12-04T05:37:58.921 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 127/165 2023-12-04T05:37:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 127/165 2023-12-04T05:37:58.943 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:58.943 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-04T05:37:58.943 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-04T05:37:58.943 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-04T05:37:58.943 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:58.979 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7531.g47c1c 128/165 2023-12-04T05:37:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7531.g47c1c 128/165 2023-12-04T05:37:59.001 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:59.001 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-04T05:37:59.001 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:59.047 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 129/165 2023-12-04T05:37:59.069 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 129/165 2023-12-04T05:37:59.069 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:59.069 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-04T05:37:59.069 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-04T05:37:59.069 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-04T05:37:59.069 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:37:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 130/165 2023-12-04T05:37:59.394 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 130/165 2023-12-04T05:37:59.394 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-04T05:37:59.395 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-04T05:37:59.395 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-04T05:37:59.395 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-04T05:37:59.395 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:03.251 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:18.0.0-7531.g47c1c547.el9.x86_64 131/165 2023-12-04T05:38:03.292 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 132/165 2023-12-04T05:38:03.332 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:18.0.0-7531.g47c1c547.el9.x86_64 133/165 2023-12-04T05:38:03.365 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 134/165 2023-12-04T05:38:03.541 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 135/165 2023-12-04T05:38:03.663 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 136/165 2023-12-04T05:38:03.696 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 137/165 2023-12-04T05:38:03.729 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:18.0.0-7531.g47c1c547.el9.x86_ 138/165 2023-12-04T05:38:03.804 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 139/165 2023-12-04T05:38:03.863 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-04T05:38:03.889 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 141/165 2023-12-04T05:38:03.920 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 142/165 2023-12-04T05:38:03.973 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 143/165 2023-12-04T05:38:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-04T05:38:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-04T05:38:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-04T05:38:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:18.0.0-7531.g47c1c547.el9.noarch 145/165 2023-12-04T05:38:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 146/165 2023-12-04T05:38:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 147/165 2023-12-04T05:38:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/165 2023-12-04T05:38:05.773 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/165 2023-12-04T05:38:05.813 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/165 2023-12-04T05:38:05.838 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/165 2023-12-04T05:38:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 152/165 2023-12-04T05:38:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 153/165 2023-12-04T05:38:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-04T05:38:05.999 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-04T05:38:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 155/165 2023-12-04T05:38:06.057 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 156/165 2023-12-04T05:38:06.083 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 157/165 2023-12-04T05:38:06.083 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-04T05:38:06.167 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-04T05:38:06.194 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-04T05:38:06.198 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-04T05:38:06.199 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-04T05:38:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-04T05:38:06.378 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-04T05:38:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-04T05:38:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 162/165 2023-12-04T05:38:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 163/165 2023-12-04T05:38:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 164/165 2023-12-04T05:38:06.618 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-04T05:38:11.115 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-04T05:38:11.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:18.0.0-7531.g47c1c547.el9.x86_64 1/165 2023-12-04T05:38:11.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 2/165 2023-12-04T05:38:11.115 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 3/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 4/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7531.g47c1c 5/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 6/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 7/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 8/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 9/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 10/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 11/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:18.0.0-7531.g47c1c547.el9.x86_64 12/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 13/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:18.0.0-7531.g47c1c547.el9.x86_ 14/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 15/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 16/165 2023-12-04T05:38:11.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:18.0.0-7531.g47c1c547.el9.x86_6 17/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86 18/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 19/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7531.g47c1c547.el 20/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:18.0.0-7531.g47c1c547.el9. 21/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:18.0.0-7531.g47c1c547.el9.x86_6 22/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:18.0.0-7531.g47c1c547.el9.x86_64 23/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:18.0.0-7531.g47c1c547.el9.x86_64 24/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:18.0.0-7531.g47c1c547.el9.x86_64 25/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 26/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 27/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:18.0.0-7531.g47c1c547.el9.x86_64 28/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7531.g47c1c547. 29/165 2023-12-04T05:38:11.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noa 30/165 2023-12-04T05:38:11.129 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.n 31/165 2023-12-04T05:38:11.129 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c 32/165 2023-12-04T05:38:11.129 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7531.g47c1c547.el 33/165 2023-12-04T05:38:11.129 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 34/165 2023-12-04T05:38:11.129 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7531.g47c1c547.e 35/165 2023-12-04T05:38:11.129 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 36/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 37/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:18.0.0-7531.g47c1c547.el9.noarch 38/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-16.el9.x86_64 47/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/165 2023-12-04T05:38:11.130 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/165 2023-12-04T05:38:11.131 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/165 2023-12-04T05:38:11.132 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/165 2023-12-04T05:38:11.132 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/165 2023-12-04T05:38:11.132 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/165 2023-12-04T05:38:11.132 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/165 2023-12-04T05:38:11.132 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/165 2023-12-04T05:38:11.132 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/165 2023-12-04T05:38:11.133 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/165 2023-12-04T05:38:11.134 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 94/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 98/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 99/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 103/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/165 2023-12-04T05:38:11.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 110/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/165 2023-12-04T05:38:11.136 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/165 2023-12-04T05:38:11.137 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 136/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 138/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-4.el9.x86_64 142/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-3.el9.x86_64 143/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 144/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 146/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 147/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 148/165 2023-12-04T05:38:11.138 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 149/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 154/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 156/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 158/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 160/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/165 2023-12-04T05:38:11.139 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/165 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/165 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: acl-2.3.1-4.el9.x86_64 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-04T05:38:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:18.0.0-7531.g47c1c547.el9.noarch 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2023-12-04T05:38:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-16.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-04T05:38:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-12-04T05:38:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.11-1.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-04T05:38:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-04T05:38:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-04T05:38:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-04T05:38:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:18.0.0-7531.g47c1c547.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-04T05:38:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-04T05:38:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-04T05:38:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:13.849 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-04T05:38:14.342 DEBUG:teuthology.parallel:result is None 2023-12-04T05:38:14.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-12-04T05:38:14.483 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-04T05:38:14.510 INFO:teuthology.orchestra.run.smithi111.stdout:18.0.0-7531.g47c1c547.el9 2023-12-04T05:38:14.510 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7531.g47c1c547.el9 2023-12-04T05:38:14.510 INFO:teuthology.task.install:The correct ceph version 18.0.0-7531.g47c1c547 is installed. 2023-12-04T05:38:14.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47c1c547372ed5bff7984647dc200bac0e019caf 2023-12-04T05:38:14.745 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-04T05:38:14.767 INFO:teuthology.orchestra.run.smithi192.stdout:18.0.0-7531.g47c1c547.el9 2023-12-04T05:38:14.767 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7531.g47c1c547.el9 2023-12-04T05:38:14.767 INFO:teuthology.task.install:The correct ceph version 18.0.0-7531.g47c1c547 is installed. 2023-12-04T05:38:14.767 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-04T05:38:14.767 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:14.767 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-04T05:38:14.803 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:14.803 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-04T05:38:14.837 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-04T05:38:14.837 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:14.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2023-12-04T05:38:14.873 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-04T05:38:14.942 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:14.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2023-12-04T05:38:14.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-04T05:38:15.038 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-04T05:38:15.038 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:15.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-04T05:38:15.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-04T05:38:15.136 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:15.136 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-04T05:38:15.163 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-04T05:38:15.229 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-04T05:38:15.229 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:15.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2023-12-04T05:38:15.258 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-04T05:38:15.326 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:15.326 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2023-12-04T05:38:15.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-04T05:38:15.420 INFO:teuthology.run_tasks:Running task ceph... 2023-12-04T05:38:15.491 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-04T05:38:15.491 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2023-12-04T05:38:15.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2023-12-04T05:38:15.520 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-04T05:38:15.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-04T05:38:15.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-04T05:38:15.589 INFO:tasks.ceph:Creating extra log directories... 2023-12-04T05:38:15.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-04T05:38:15.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-04T05:38:15.663 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-04T05:38:15.663 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-04T05:38:15.663 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-04T05:38:15.664 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:15.664 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-04T05:38:15.711 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-04T05:38:15.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-04T05:38:15.851 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:15.852 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-04T05:38:15.866 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-04T05:38:15.985 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-04T05:38:16.017 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-04T05:38:16.017 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'client session with non-allowable root'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-04T05:38:16.017 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641', 'branch': 'wip-vshankar-testing-20231127.102654', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7476641', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20231127.102654'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'client session with non-allowable root'], 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20231127.102654', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1098, 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'sleep_before_teardown': 0, 'subset': '854/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20231127.102654', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi111.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP7TOh+HZ3oyc1UE1dEmLMraqDwNhhMUnk6LiwOPFMRjk+UfO70Y+SWF1YQSouzAW6QbWiogrnTHE8H7oEPYl2U=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQEz4rvWx4sxcwW76PgSTUPMID8HIMA8zxtTk/Wv1482QciO4OcN7Sh7KpO2C9zAeVTY+hosTMmRVjHxmy5GEY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'client session with non-allowable root'], 'sha1': '47c1c547372ed5bff7984647dc200bac0e019caf', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_sessionmap']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'f2de108beb0a6dde7f3e6c96039071828a1f6269', 'timestamp': '2023-12-04_05:06:51', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-12-04T05:38:16.018 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-04T05:38:16.020 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-04T05:38:16.066 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2023-12-04T05:38:16.075 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2023-12-04T05:38:16.136 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:16.137 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2023-12-04T05:38:16.155 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-04T05:38:16.156 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 664 Links: 1 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-04 05:37:49.740973262 +0000 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-04 05:35:00.619603143 +0000 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-04 05:35:00.619603143 +0000 2023-12-04T05:38:16.211 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-12-04 05:35:00.619603143 +0000 2023-12-04T05:38:16.212 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-04T05:38:16.278 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-04T05:38:16.278 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-04T05:38:16.278 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000186841 s, 2.7 MB/s 2023-12-04T05:38:16.279 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-04T05:38:16.335 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 681 Links: 1 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-04 05:37:49.740973262 +0000 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-04 05:35:01.520587086 +0000 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-04 05:35:01.520587086 +0000 2023-12-04T05:38:16.391 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-12-04 05:35:01.520587086 +0000 2023-12-04T05:38:16.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-04T05:38:16.456 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-04T05:38:16.456 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-04T05:38:16.456 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000219049 s, 2.3 MB/s 2023-12-04T05:38:16.457 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-04T05:38:16.514 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2023-12-04T05:38:16.568 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 701 Links: 1 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-04 05:37:49.741973245 +0000 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-04 05:35:02.447570681 +0000 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-04 05:35:02.447570681 +0000 2023-12-04T05:38:16.569 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-12-04 05:35:02.447570681 +0000 2023-12-04T05:38:16.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-04T05:38:16.635 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-04T05:38:16.635 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-04T05:38:16.635 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000202143 s, 2.5 MB/s 2023-12-04T05:38:16.636 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-04T05:38:16.691 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 722 Links: 1 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-04 05:37:49.741973245 +0000 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-04 05:35:03.240556649 +0000 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-04 05:35:03.240556649 +0000 2023-12-04T05:38:16.747 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-12-04 05:35:03.240556649 +0000 2023-12-04T05:38:16.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-04T05:38:16.814 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-04T05:38:16.829 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-04T05:38:16.830 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000163828 s, 3.1 MB/s 2023-12-04T05:38:16.830 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-04T05:38:16.870 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:16.871 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:16.871 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2023-12-04T05:38:16.887 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-04T05:38:16.887 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-04 05:38:06.957492376 +0000 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-04 05:35:00.625156362 +0000 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-04 05:35:00.625156362 +0000 2023-12-04T05:38:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-04 05:35:00.625156362 +0000 2023-12-04T05:38:16.945 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-04T05:38:17.010 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-04T05:38:17.011 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-04T05:38:17.011 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000273624 s, 1.9 MB/s 2023-12-04T05:38:17.012 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-04T05:38:17.068 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2023-12-04T05:38:17.124 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-04T05:38:17.124 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:17.124 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-12-04T05:38:17.124 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:17.124 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:17.124 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-04 05:38:06.957492376 +0000 2023-12-04T05:38:17.125 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-04 05:35:01.481139455 +0000 2023-12-04T05:38:17.125 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-04 05:35:01.481139455 +0000 2023-12-04T05:38:17.125 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-04 05:35:01.481139455 +0000 2023-12-04T05:38:17.125 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-04T05:38:17.190 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-04T05:38:17.191 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-04T05:38:17.191 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000214457 s, 2.4 MB/s 2023-12-04T05:38:17.192 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-04T05:38:17.248 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-04 05:38:06.957492376 +0000 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-04 05:35:02.426120797 +0000 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-04 05:35:02.426120797 +0000 2023-12-04T05:38:17.303 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-04 05:35:02.426120797 +0000 2023-12-04T05:38:17.304 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-04T05:38:17.368 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-04T05:38:17.369 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-04T05:38:17.369 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000203005 s, 2.5 MB/s 2023-12-04T05:38:17.370 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-04T05:38:17.425 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-04 05:38:06.958492357 +0000 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-04 05:35:03.361102336 +0000 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-04 05:35:03.361102336 +0000 2023-12-04T05:38:17.481 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-04 05:35:03.361102336 +0000 2023-12-04T05:38:17.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-04T05:38:17.546 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-04T05:38:17.546 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-04T05:38:17.546 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000234333 s, 2.2 MB/s 2023-12-04T05:38:17.547 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-04T05:38:17.603 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:17.603 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-04T05:38:17.604 INFO:tasks.ceph:Generating config... 2023-12-04T05:38:17.605 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-04T05:38:17.605 INFO:tasks.ceph:[client] debug client = 20 2023-12-04T05:38:17.605 INFO:tasks.ceph:[client] debug ms = 1 2023-12-04T05:38:17.605 INFO:tasks.ceph:[client] pid file = 2023-12-04T05:38:17.605 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-04T05:38:17.605 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-04T05:38:17.605 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-04T05:38:17.605 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-04T05:38:17.605 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-04T05:38:17.605 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-04T05:38:17.605 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-04T05:38:17.605 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-04T05:38:17.606 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-04T05:38:17.606 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-04T05:38:17.606 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-04T05:38:17.606 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-04T05:38:17.606 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-04T05:38:17.607 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-04T05:38:17.607 INFO:tasks.ceph:Setting up mon.a... 2023-12-04T05:38:17.608 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-04T05:38:17.689 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.keyring 2023-12-04T05:38:17.692 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-04T05:38:17.737 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-04T05:38:17.764 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.111'), ('mon.b', '172.21.15.192'), ('mon.c', '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]')] 2023-12-04T05:38:17.765 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.111,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-04T05:38:17.766 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:17.766 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-04T05:38:17.820 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.111 --add b 172.21.15.192 --addv c '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:17.906 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: generated fsid 9686316e-a01b-4981-b0b7-e21bdb185d5e 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:setting min_mon_release = pacific 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 0 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:fsid 9686316e-a01b-4981-b0b7-e21bdb185d5e 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:last_changed 2023-12-04T05:38:17.904535+0000 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:created 2023-12-04T05:38:17.904535+0000 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:min_mon_release 16 (pacific) 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:election_strategy: 1 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:2: [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] mon.c 2023-12-04T05:38:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-04T05:38:17.908 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-04T05:38:17.922 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9686316e-a01b-4981-b0b7-e21bdb185d5e... 2023-12-04T05:38:17.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-04T05:38:17.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-04T05:38:18.012 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2023-12-04T05:38:18.012 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2023-12-04T05:38:18.012 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2023-12-04T05:38:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-04T05:38:18.014 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.111,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = 9686316e-a01b-4981-b0b7-e21bdb185d5e 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.015 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 600 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock profile = high_recovery_ops 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug trim objects = true 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2023-12-04T05:38:18.016 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2023-12-04T05:38:18.017 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: bdev async discard = True 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: bdev enable discard = True 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore allocator = bitmap 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 1/20 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 1/20 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-12-04T05:38:18.018 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 4/10 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2023-12-04T05:38:18.019 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-12-04T05:38:18.020 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2023-12-04T05:38:18.021 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2023-12-04T05:38:18.022 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2023-12-04T05:38:18.022 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.022 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-04T05:38:18.022 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2023-12-04T05:38:18.022 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2023-12-04T05:38:18.022 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2023-12-04T05:38:18.023 INFO:teuthology.orchestra.run.smithi111.stdout: mon op complaint time = 120 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-04T05:38:18.024 INFO:teuthology.orchestra.run.smithi111.stdout: client mount timeout = 600 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = "" 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2023-12-04T05:38:18.025 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout:[mds] 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds = 20 2023-12-04T05:38:18.026 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds balancer = 20 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug frag = True 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug scatterstat = True 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: mds op complaint time = 180 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: mds verify scatter = True 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2023-12-04T05:38:18.027 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2023-12-04T05:38:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2023-12-04T05:38:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2023-12-04T05:38:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.111,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 9686316e-a01b-4981-b0b7-e21bdb185d5e 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2023-12-04T05:38:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2023-12-04T05:38:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: bdev async discard = True 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: bdev enable discard = True 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore allocator = bitmap 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2023-12-04T05:38:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2023-12-04T05:38:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2023-12-04T05:38:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = "" 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2023-12-04T05:38:18.037 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds balancer = 20 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2023-12-04T05:38:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2023-12-04T05:38:18.040 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-04T05:38:18.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-04T05:38:18.135 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-04T05:38:18.135 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:18.135 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-04T05:38:18.150 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:18.150 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-04T05:38:18.205 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:38:18.205 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:18.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-04T05:38:18.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-04T05:38:18.283 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:18.283 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:18.337 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:38:18.338 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:18.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-04T05:38:18.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-04T05:38:18.378 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:18.378 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:18.433 INFO:tasks.ceph:Setting up mon nodes... 2023-12-04T05:38:18.433 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-04T05:38:18.433 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-04T05:38:18.488 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-04T05:38:18.490 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-04T05:38:18.557 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-04T05:38:18.559 INFO:tasks.ceph:Setting up mds nodes... 2023-12-04T05:38:18.559 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-04T05:38:18.614 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-04T05:38:18.616 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-04T05:38:18.643 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-04T05:38:18.737 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-04T05:38:18.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-04T05:38:18.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-04T05:38:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-04T05:38:18.824 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-04T05:38:18.851 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-04T05:38:18.945 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-04T05:38:18.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-04T05:38:18.974 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-04T05:38:18.974 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-04T05:38:19.016 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-04T05:38:19.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-04T05:38:19.114 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-04T05:38:19.129 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-04T05:38:19.172 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-04T05:38:19.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-04T05:38:19.270 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-04T05:38:19.285 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-04T05:38:19.285 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-04T05:38:19.285 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-04T05:38:19.312 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:19.312 INFO:tasks.ceph:role: osd.0 2023-12-04T05:38:19.313 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:38:19.313 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-04T05:38:19.386 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:19.386 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:19.386 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:19.387 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:19.892 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-12-04T05:38:19.894 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-12-04T05:38:19.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-04T05:38:19.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-04T05:38:20.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-04T05:38:20.101 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:20.101 INFO:tasks.ceph:role: osd.1 2023-12-04T05:38:20.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:38:20.101 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-04T05:38:20.175 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:20.175 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:20.175 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:20.176 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:20.633 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-12-04T05:38:20.635 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-12-04T05:38:20.635 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-04T05:38:20.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-04T05:38:20.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-04T05:38:20.826 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:20.826 INFO:tasks.ceph:role: osd.2 2023-12-04T05:38:20.826 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:38:20.826 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:20.901 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:21.384 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-12-04T05:38:21.386 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-12-04T05:38:21.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-04T05:38:21.525 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-04T05:38:21.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-04T05:38:21.625 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:21.625 INFO:tasks.ceph:role: osd.3 2023-12-04T05:38:21.625 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:38:21.625 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-04T05:38:21.699 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:21.699 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:21.699 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:21.699 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:21.699 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:21.699 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:21.700 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:21.700 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:21.700 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:21.700 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:22.224 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-12-04T05:38:22.226 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-12-04T05:38:22.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-04T05:38:22.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-04T05:38:22.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:22.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:22.497+0000 7f97d1410500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-04T05:38:22.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:22.497+0000 7f97d1410500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-04T05:38:22.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:22.497+0000 7f97d1410500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-04T05:38:22.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:22.497+0000 7f97d1410500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-04T05:38:22.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:22.497+0000 7f97d1410500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-04T05:38:22.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:22.498+0000 7f97d1410500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-04T05:38:25.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-04T05:38:25.401 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:25.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:25.492+0000 7f36f2625500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-04T05:38:25.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:25.492+0000 7f36f2625500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-04T05:38:25.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:25.492+0000 7f36f2625500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-04T05:38:25.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:25.492+0000 7f36f2625500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-04T05:38:25.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:25.492+0000 7f36f2625500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-04T05:38:25.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:25.493+0000 7f36f2625500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-04T05:38:28.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-04T05:38:28.374 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:28.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:28.463+0000 7fe79f6d4500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-04T05:38:28.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:28.463+0000 7fe79f6d4500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-04T05:38:28.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:28.463+0000 7fe79f6d4500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-04T05:38:28.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:28.463+0000 7fe79f6d4500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-04T05:38:28.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:28.463+0000 7fe79f6d4500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-04T05:38:28.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:28.464+0000 7fe79f6d4500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-04T05:38:31.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-04T05:38:31.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:31.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:31.422+0000 7f4088a0d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-04T05:38:31.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:31.422+0000 7f4088a0d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-04T05:38:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:31.422+0000 7f4088a0d500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-04T05:38:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:31.422+0000 7f4088a0d500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-04T05:38:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:31.422+0000 7f4088a0d500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-04T05:38:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:31.422+0000 7f4088a0d500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-04T05:38:34.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-04T05:38:34.304 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-04T05:38:34.337 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:34.337 INFO:tasks.ceph:role: osd.4 2023-12-04T05:38:34.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:38:34.337 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:34.415 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:34.416 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:34.416 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:34.923 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-04T05:38:34.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-04T05:38:34.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-04T05:38:35.060 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-04T05:38:35.092 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-04T05:38:35.159 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:35.159 INFO:tasks.ceph:role: osd.5 2023-12-04T05:38:35.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:38:35.159 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-04T05:38:35.233 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:35.233 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:35.234 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:35.694 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-04T05:38:35.696 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-04T05:38:35.696 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-04T05:38:35.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-04T05:38:35.853 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-04T05:38:35.920 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:35.920 INFO:tasks.ceph:role: osd.6 2023-12-04T05:38:35.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:38:35.920 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-04T05:38:35.994 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:35.995 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:36.508 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-04T05:38:36.510 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-04T05:38:36.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-04T05:38:36.664 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-04T05:38:36.696 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-04T05:38:36.763 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-04T05:38:36.764 INFO:tasks.ceph:role: osd.7 2023-12-04T05:38:36.764 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:38:36.764 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-04T05:38:36.838 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-04T05:38:37.341 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-04T05:38:37.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-04T05:38:37.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-04T05:38:37.427 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-04T05:38:37.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:37.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:37.621+0000 7fd09349a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-04T05:38:37.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:37.621+0000 7fd09349a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-04T05:38:37.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:37.621+0000 7fd09349a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-04T05:38:37.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:37.621+0000 7fd09349a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-04T05:38:37.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:37.621+0000 7fd09349a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-04T05:38:37.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:37.622+0000 7fd09349a500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-04T05:38:40.479 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-04T05:38:40.511 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:40.605 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:40.604+0000 7f9ce7620500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-04T05:38:40.606 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:40.604+0000 7f9ce7620500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-04T05:38:40.606 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:40.604+0000 7f9ce7620500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-04T05:38:40.606 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:40.604+0000 7f9ce7620500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-04T05:38:40.606 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:40.604+0000 7f9ce7620500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-04T05:38:40.607 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:40.605+0000 7f9ce7620500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-04T05:38:43.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-04T05:38:43.479 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:43.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:43.569+0000 7fdad114a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-04T05:38:43.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:43.569+0000 7fdad114a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-04T05:38:43.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:43.569+0000 7fdad114a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-04T05:38:43.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:43.569+0000 7fdad114a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-04T05:38:43.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:43.569+0000 7fdad114a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-04T05:38:43.571 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:43.570+0000 7fdad114a500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-04T05:38:46.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:38:46.022 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:38:46.414 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-04T05:38:46.441 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:46.529 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:46.527+0000 7f45e961a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-04T05:38:46.529 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:46.527+0000 7f45e961a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-04T05:38:46.529 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:46.527+0000 7f45e961a500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-04T05:38:46.529 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:46.527+0000 7f45e961a500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-04T05:38:46.529 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:46.527+0000 7f45e961a500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-04T05:38:46.529 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:46.528+0000 7f45e961a500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-04T05:38:49.393 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-04T05:38:49.424 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-04T05:38:49.424 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-04T05:38:49.455 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-04T05:38:49.526 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.526 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-04T05:38:49.595 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-04T05:38:49.663 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.663 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-04T05:38:49.730 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-04T05:38:49.798 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:49.798 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-04T05:38:49.864 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:49.865 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-04T05:38:49.896 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:49.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-04T05:38:49.966 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:49.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-04T05:38:50.037 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:50.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-04T05:38:50.108 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:50.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-04T05:38:50.177 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:50.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-04T05:38:50.246 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:50.246 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-04T05:38:50.314 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:50.314 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-04T05:38:50.330 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:50.331 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-04T05:38:50.387 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:50.387 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-04T05:38:50.403 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:50.403 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-04T05:38:50.458 INFO:tasks.ceph:Adding keys to all mons... 2023-12-04T05:38:50.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-04T05:38:50.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.y] 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDKZW1lztwVHRAAHxYfbr/N8aCdSCIPzjqzXw== 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDKZW1lpb6cJBAA7iF+mExHXiopk7JbrjCW1g== 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDKZW1lgo37KxAA/89DL0NuA+KyX+CUbzzU+A== 2023-12-04T05:38:50.502 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDOZW1lxDi6HRAA0pl+LbQDgGxT9fr9vGzWHA== 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDRZW1lu/JsHRAApgnvWoeQ9MAOPdBuFifdvA== 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDUZW1ldy+tGxAAhPf6cR9pyGpyD0cAy1nd5g== 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDXZW1lDCs7GRAAADDCyZOB4a+fmmWEgezmxw== 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDKZW1l8LA1IRAAZmq/pSux9bgsmwY5LTOzIQ== 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDKZW1l+X3+MBAAQatwhQBZ5/VZ0xuNJpQSYA== 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2023-12-04T05:38:50.503 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDKZW1lPSdXOBAAGSdsougjKMP1afwkK/2gmw== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDdZW1lzq4XJRAAG90v2DSwzGMnGUEK14Z7ew== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDgZW1lvzQUJBAAYrLHQV/+N0irpVvSJh1LkQ== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDjZW1ll6sCIhAAlUiDCgD51TZDbjgznyHYSw== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDmZW1ll2Z/HxAALDVVTrBx3aaiRVf/Is2p5Q== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDLZW1l+5T3ABAA7h+0Ksz9xnrfFZKgsBbhIA== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[client.2] 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDLZW1lwejPBhAAVmTVqunSjtPrXqLWy4n1AA== 2023-12-04T05:38:50.504 INFO:teuthology.orchestra.run.smithi111.stdout:[client.1] 2023-12-04T05:38:50.505 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDLZW1lqipMChAAXB0Z40yv2jFsOlHuH8i4Pg== 2023-12-04T05:38:50.505 INFO:teuthology.orchestra.run.smithi111.stdout:[client.3] 2023-12-04T05:38:50.505 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDLZW1lrQobEBAAAJ8wvchn2R7AvCfLR5Me0w== 2023-12-04T05:38:50.524 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2023-12-04T05:38:50.524 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDKZW1lztwVHRAAHxYfbr/N8aCdSCIPzjqzXw== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDKZW1lpb6cJBAA7iF+mExHXiopk7JbrjCW1g== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDKZW1lgo37KxAA/89DL0NuA+KyX+CUbzzU+A== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDOZW1lxDi6HRAA0pl+LbQDgGxT9fr9vGzWHA== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDRZW1lu/JsHRAApgnvWoeQ9MAOPdBuFifdvA== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDUZW1ldy+tGxAAhPf6cR9pyGpyD0cAy1nd5g== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDXZW1lDCs7GRAAADDCyZOB4a+fmmWEgezmxw== 2023-12-04T05:38:50.525 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDKZW1l8LA1IRAAZmq/pSux9bgsmwY5LTOzIQ== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDKZW1l+X3+MBAAQatwhQBZ5/VZ0xuNJpQSYA== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDKZW1lPSdXOBAAGSdsougjKMP1afwkK/2gmw== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDdZW1lzq4XJRAAG90v2DSwzGMnGUEK14Z7ew== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDgZW1lvzQUJBAAYrLHQV/+N0irpVvSJh1LkQ== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDjZW1ll6sCIhAAlUiDCgD51TZDbjgznyHYSw== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDmZW1ll2Z/HxAALDVVTrBx3aaiRVf/Is2p5Q== 2023-12-04T05:38:50.526 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDLZW1l+5T3ABAA7h+0Ksz9xnrfFZKgsBbhIA== 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout:[client.2] 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDLZW1lwejPBhAAVmTVqunSjtPrXqLWy4n1AA== 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDLZW1lqipMChAAXB0Z40yv2jFsOlHuH8i4Pg== 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout:[client.3] 2023-12-04T05:38:50.527 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDLZW1lrQobEBAAAJ8wvchn2R7AvCfLR5Me0w== 2023-12-04T05:38:50.528 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-04T05:38:50.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-04T05:38:50.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:50.622 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:50.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:50.679 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:50.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.843 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.898 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:50.950 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-04T05:38:50.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-04T05:38:51.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:51.006 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:51.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:51.062 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-04T05:38:51.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-04T05:38:51.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.332 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.388 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-04T05:38:51.556 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-04T05:38:51.556 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-04T05:38:51.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-04T05:38:51.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-04T05:38:51.705 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-04T05:38:51.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-04T05:38:51.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-04T05:38:51.856 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-04T05:38:51.924 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-04T05:38:52.021 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-04T05:38:52.047 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:52.050 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-04T05:38:52.102 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-04T05:38:52.103 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-04T05:38:52.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-04T05:38:52.105 INFO:tasks.ceph.mon.a:Started 2023-12-04T05:38:52.105 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-04T05:38:52.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-04T05:38:52.145 INFO:tasks.ceph.mon.b:Started 2023-12-04T05:38:52.145 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-04T05:38:52.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-04T05:38:52.147 INFO:tasks.ceph.mon.c:Started 2023-12-04T05:38:52.147 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-04T05:38:52.147 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-04T05:38:52.147 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-04T05:38:52.149 INFO:tasks.ceph.mgr.y:Started 2023-12-04T05:38:52.150 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-04T05:38:52.150 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-04T05:38:52.152 INFO:tasks.ceph.mgr.x:Started 2023-12-04T05:38:52.152 INFO:tasks.ceph:Setting crush tunables to default 2023-12-04T05:38:52.152 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd crush tunables default 2023-12-04T05:38:52.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.252+0000 7f105593e640 1 Processor -- start 2023-12-04T05:38:52.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.252+0000 7f105593e640 1 -- start start 2023-12-04T05:38:52.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f105593e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f105593e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f105593e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f105593e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105015ddc0 con 0x7f105015d480 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f105593e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105015e0a0 con 0x7f105015d860 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f105593e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105015e380 con 0x7f10500d1310 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41224/0 (socket says 172.21.15.111:41224) 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 -- 172.21.15.111:0/966742441 learned_addr learned my addr 172.21.15.111:0/966742441 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:38:52.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104f7fe640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 -- 172.21.15.111:0/966742441 >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 msgr2=0x7f10500d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 -- 172.21.15.111:0/966742441 >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 msgr2=0x7f10500d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f1047fff640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104effd640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f1047fff640 1 -- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 msgr2=0x7f10500cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-04T05:38:52.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f1047fff640 1 -- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 msgr2=0x7f10500cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-04T05:38:52.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f1047fff640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-04T05:38:52.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f1047fff640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:38:52.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104f7fe640 1 -- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 msgr2=0x7f10500d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-04T05:38:52.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104f7fe640 1 -- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 msgr2=0x7f10500d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-04T05:38:52.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104f7fe640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-04T05:38:52.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.253+0000 7f104f7fe640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:38:52.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104effd640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.111:3300/0 conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104f7fe640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f1047fff640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104effd640 1 -- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 msgr2=0x7f10500cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104effd640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104effd640 1 -- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 msgr2=0x7f10500d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104effd640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104effd640 1 -- 172.21.15.111:0/966742441 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105015e660 con 0x7f105015d480 2023-12-04T05:38:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.454+0000 7f104f7fe640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f104effd640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105015d480 0x7f10500d0dd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f105008d3f0 tx=0x7f104002bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226e8c9085e339e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f104e7fc640 1 -- 172.21.15.111:0/966742441 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104002ca20 con 0x7f105015d480 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f104e7fc640 1 -- 172.21.15.111:0/966742441 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104002cb80 con 0x7f105015d480 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f104e7fc640 1 -- 172.21.15.111:0/966742441 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040034440 con 0x7f105015d480 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 -- 172.21.15.111:0/966742441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105015d480 msgr2=0x7f10500d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105015d480 0x7f10500d0dd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f105008d3f0 tx=0x7f104002bc70 comp rx=0 tx=0).stop 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 -- 172.21.15.111:0/966742441 shutdown_connections 2023-12-04T05:38:52.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 --2- 172.21.15.111:0/966742441 >> v2:172.21.15.192:3300/0 conn(0x7f105015d860 0x7f10500d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10500d1310 0x7f10500cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 --2- 172.21.15.111:0/966742441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f105015d480 0x7f10500d0dd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.455+0000 7f105593e640 1 -- 172.21.15.111:0/966742441 >> 172.21.15.111:0/966742441 conn(0x7f105009c540 msgr2=0x7f10500d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.456+0000 7f105593e640 1 -- 172.21.15.111:0/966742441 shutdown_connections 2023-12-04T05:38:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.456+0000 7f105593e640 1 -- 172.21.15.111:0/966742441 wait complete. 2023-12-04T05:38:52.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 Processor -- start 2023-12-04T05:38:52.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 -- start start 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 0x7f10500b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105015d480 0x7f10501463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d860 0x7f10500b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500b78d0 con 0x7f10500d1310 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050159bb0 con 0x7f105015d480 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.457+0000 7f105593e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1050159e50 con 0x7f105015d860 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 0x7f10500b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 0x7f10500b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41254/0 (socket says 172.21.15.111:41254) 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 -- 172.21.15.111:0/740027023 learned_addr learned my addr 172.21.15.111:0/740027023 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f1047fff640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105015d480 0x7f10501463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104f7fe640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d860 0x7f10500b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:52.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 -- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d860 msgr2=0x7f10500b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d860 0x7f10500b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 -- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105015d480 msgr2=0x7f10501463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105015d480 0x7f10501463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 -- 172.21.15.111:0/740027023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048009ad0 con 0x7f10500d1310 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f1047fff640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105015d480 0x7f10501463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104f7fe640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d860 0x7f10500b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104effd640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 0x7f10500b72d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1040037040 tx=0x7f1040004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040003a20 con 0x7f10500d1310 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1040003b80 con 0x7f10500d1310 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10400095b0 con 0x7f10500d1310 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400044e0 con 0x7f10500d1310 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1050156c90 con 0x7f10500d1310 2023-12-04T05:38:52.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f104003b050 con 0x7f10500d1310 2023-12-04T05:38:52.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.458+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1040034780 con 0x7f10500d1310 2023-12-04T05:38:52.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.459+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1050067980 con 0x7f10500d1310 2023-12-04T05:38:52.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.461+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f104004e060 con 0x7f10500d1310 2023-12-04T05:38:52.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.526+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1050069360 con 0x7f10500d1310 2023-12-04T05:38:52.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.528+0000 7f104cff9640 1 -- 172.21.15.111:0/740027023 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1040044070 con 0x7f10500d1310 2023-12-04T05:38:52.529 INFO:teuthology.orchestra.run.smithi111.stderr:adjusted tunables profile to default 2023-12-04T05:38:52.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 msgr2=0x7f10500b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:52.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 0x7f10500b72d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1040037040 tx=0x7f1040004730 comp rx=0 tx=0).stop 2023-12-04T05:38:52.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 shutdown_connections 2023-12-04T05:38:52.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d860 0x7f10500b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f105015d480 0x7f10501463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 --2- 172.21.15.111:0/740027023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10500d1310 0x7f10500b72d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:52.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 >> 172.21.15.111:0/740027023 conn(0x7f105009c540 msgr2=0x7f1050160800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:52.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 shutdown_connections 2023-12-04T05:38:52.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:52.529+0000 7f105593e640 1 -- 172.21.15.111:0/740027023 wait complete. 2023-12-04T05:38:52.541 INFO:tasks.ceph:check_enable_crimson: False 2023-12-04T05:38:52.541 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-04T05:38:52.541 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:52.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-04T05:38:52.568 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:52.568 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-04T05:38:52.635 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:52.635 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-04T05:38:52.702 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:38:52.702 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-04T05:38:52.769 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:52.769 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-04T05:38:52.796 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:52.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-04T05:38:52.864 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:52.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-04T05:38:52.888 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:52.887+0000 7f14d0213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-04T05:38:52.906 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:52.905+0000 7f868bc39040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-04T05:38:52.915 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:38:52.915 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-04T05:38:52.943 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:52.942+0000 7f14d0213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-04T05:38:52.956 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:52.955+0000 7f868bc39040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-04T05:38:52.970 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 130f386c-483a-4a82-9a0e-1a94f399374e 0 2023-12-04T05:38:53.111 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.110+0000 7fd4c77a9640 1 Processor -- start 2023-12-04T05:38:53.111 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.110+0000 7fd4c77a9640 1 -- start start 2023-12-04T05:38:53.112 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c77a9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4c01438f0 0x7fd4c00cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.112 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c77a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4c0071520 0x7fd4c00719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c77a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c00724c0 con 0x7fd4c0143390 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c77a9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c0143770 con 0x7fd4c01438f0 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c77a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c0071ef0 con 0x7fd4c0071520 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4c0071520 0x7fd4c00719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4c0071520 0x7fd4c00719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38930/0 (socket says 172.21.15.192:38930) 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 -- 172.21.15.192:0/985598105 learned_addr learned my addr 172.21.15.192:0/985598105 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5545640 1 --1- 172.21.15.192:0/985598105 >> v1:172.21.15.192:6789/0 conn(0x7fd4c0143390 0x7fd4c00cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56374/0 (socket says 172.21.15.192:56374) 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c4d44640 1 --2- 172.21.15.192:0/985598105 >> v2:172.21.15.111:3300/0 conn(0x7fd4c01438f0 0x7fd4c00cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 -- 172.21.15.192:0/985598105 >> v2:172.21.15.111:3300/0 conn(0x7fd4c01438f0 msgr2=0x7fd4c00cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 --2- 172.21.15.192:0/985598105 >> v2:172.21.15.111:3300/0 conn(0x7fd4c01438f0 0x7fd4c00cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 -- 172.21.15.192:0/985598105 >> v1:172.21.15.192:6789/0 conn(0x7fd4c0143390 legacy=0x7fd4c00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.113 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.111+0000 7fd4c5d46640 1 -- 172.21.15.192:0/985598105 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00729f0 con 0x7fd4c0071520 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.112+0000 7fd4c5d46640 1 --2- 172.21.15.192:0/985598105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0071520 0x7fd4c00719b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00afc20 tx=0x7fd4b802e180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca5cfc50143405a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.112+0000 7fd4b7fff640 1 -- 172.21.15.192:0/985598105 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800b0d0 con 0x7fd4c0071520 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.112+0000 7fd4b7fff640 1 -- 172.21.15.192:0/985598105 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b800b230 con 0x7fd4c0071520 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.112+0000 7fd4b7fff640 1 -- 172.21.15.192:0/985598105 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8009600 con 0x7fd4c0071520 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 -- 172.21.15.192:0/985598105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0071520 msgr2=0x7fd4c00719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/985598105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0071520 0x7fd4c00719b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00afc20 tx=0x7fd4b802e180 comp rx=0 tx=0).stop 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 -- 172.21.15.192:0/985598105 shutdown_connections 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/985598105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0071520 0x7fd4c00719b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/985598105 >> v2:172.21.15.111:3300/0 conn(0x7fd4c01438f0 0x7fd4c00cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 -- 172.21.15.192:0/985598105 >> 172.21.15.192:0/985598105 conn(0x7fd4c006d4a0 msgr2=0x7fd4c006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.115 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 -- 172.21.15.192:0/985598105 shutdown_connections 2023-12-04T05:38:53.116 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.113+0000 7fd4c77a9640 1 -- 172.21.15.192:0/985598105 wait complete. 2023-12-04T05:38:53.116 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.114+0000 7fd4c77a9640 1 Processor -- start 2023-12-04T05:38:53.116 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.114+0000 7fd4c77a9640 1 -- start start 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0071520 0x7fd4c0137720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 0x7fd4c013f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4c01438f0 0x7fd4c013fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0138e90 con 0x7fd4c0071520 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c013a2f0 con 0x7fd4c0143390 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c013b350 con 0x7fd4c01438f0 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 0x7fd4c013f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 0x7fd4c013f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38944/0 (socket says 172.21.15.192:38944) 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 -- 172.21.15.192:0/2731270034 learned_addr learned my addr 172.21.15.192:0/2731270034 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c5d46640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4c01438f0 0x7fd4c013fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c5545640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0071520 0x7fd4c0137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.118 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 -- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4c01438f0 msgr2=0x7fd4c013fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4c01438f0 0x7fd4c013fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 -- 172.21.15.192:0/2731270034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0071520 msgr2=0x7fd4c0137720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0071520 0x7fd4c0137720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 -- 172.21.15.192:0/2731270034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b800bd40 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c4d44640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 0x7fd4c013f840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000e9b0 tx=0x7fd4b000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0009670 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b000c9a0 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000cd50 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.115+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c013a590 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.116+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c013ab50 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.117+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd4b0013770 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.117+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd4b001cc40 con 0x7fd4c0143390 2023-12-04T05:38:53.119 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.117+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c0067460 con 0x7fd4c0143390 2023-12-04T05:38:53.120 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.119+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd4b0021060 con 0x7fd4c0143390 2023-12-04T05:38:53.192 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.190+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "130f386c-483a-4a82-9a0e-1a94f399374e", "id": 0} v 0) v1 -- 0x7fd4c013f5c0 con 0x7fd4c0143390 2023-12-04T05:38:53.197 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.196+0000 7fd4b5ffb640 1 -- 172.21.15.192:0/2731270034 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "130f386c-483a-4a82-9a0e-1a94f399374e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd4b0017070 con 0x7fd4c0143390 2023-12-04T05:38:53.197 INFO:teuthology.orchestra.run.smithi192.stdout:0 2023-12-04T05:38:53.198 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 msgr2=0x7fd4c013f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 0x7fd4c013f840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000e9b0 tx=0x7fd4b000ee70 comp rx=0 tx=0).stop 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 shutdown_connections 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4c01438f0 0x7fd4c013fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4c0143390 0x7fd4c013f840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 --2- 172.21.15.192:0/2731270034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4c0071520 0x7fd4c0137720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 >> 172.21.15.192:0/2731270034 conn(0x7fd4c006d4a0 msgr2=0x7fd4c00cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 shutdown_connections 2023-12-04T05:38:53.199 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.197+0000 7fd4c77a9640 1 -- 172.21.15.192:0/2731270034 wait complete. 2023-12-04T05:38:53.209 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 5c1fd699-00ab-4e49-bfd2-d2d3b9406819 1 2023-12-04T05:38:53.309 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.308+0000 7f68d4733640 1 Processor -- start 2023-12-04T05:38:53.309 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.308+0000 7f68d4733640 1 -- start start 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d4733640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68cc15d8c0 0x7f68cc06a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d4733640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc06aa00 0x7f68cc161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d4733640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc06ae50 con 0x7f68cc140280 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d4733640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc143a80 con 0x7f68cc15d8c0 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d4733640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc142f00 con 0x7f68cc06aa00 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68cc15d8c0 0x7f68cc06a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d2cd0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc06aa00 0x7f68cc161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68cc15d8c0 0x7f68cc06a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38946/0 (socket says 172.21.15.192:38946) 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 -- 172.21.15.192:0/2673951207 learned_addr learned my addr 172.21.15.192:0/2673951207 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 -- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc06aa00 msgr2=0x7f68cc161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 --2- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc06aa00 0x7f68cc161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 -- 172.21.15.192:0/2673951207 >> v1:172.21.15.192:6789/0 conn(0x7f68cc140280 legacy=0x7f68cc0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d1cce640 1 -- 172.21.15.192:0/2673951207 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc1431e0 con 0x7f68cc15d8c0 2023-12-04T05:38:53.312 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.309+0000 7f68d2cd0640 1 --2- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc06aa00 0x7f68cc161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.310+0000 7f68d1cce640 1 --2- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc15d8c0 0x7f68cc06a4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009870 tx=0x7f68c802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75aed236fa1ca9db server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.310+0000 7f68d14cd640 1 -- 172.21.15.192:0/2673951207 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8031640 con 0x7f68cc15d8c0 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.310+0000 7f68d14cd640 1 -- 172.21.15.192:0/2673951207 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c80317a0 con 0x7f68cc15d8c0 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.310+0000 7f68d14cd640 1 -- 172.21.15.192:0/2673951207 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8031b30 con 0x7f68cc15d8c0 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 -- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc15d8c0 msgr2=0x7f68cc06a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 --2- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc15d8c0 0x7f68cc06a4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009870 tx=0x7f68c802b8f0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 -- 172.21.15.192:0/2673951207 shutdown_connections 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 --2- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc06aa00 0x7f68cc161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 --2- 172.21.15.192:0/2673951207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc15d8c0 0x7f68cc06a4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 -- 172.21.15.192:0/2673951207 >> 172.21.15.192:0/2673951207 conn(0x7f68cc0b1de0 msgr2=0x7f68cc0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 -- 172.21.15.192:0/2673951207 shutdown_connections 2023-12-04T05:38:53.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.311+0000 7f68d4733640 1 -- 172.21.15.192:0/2673951207 wait complete. 2023-12-04T05:38:53.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.313+0000 7f68d4733640 1 Processor -- start 2023-12-04T05:38:53.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.313+0000 7f68d4733640 1 -- start start 2023-12-04T05:38:53.318 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d4733640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 0x7f68cc14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d4733640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68cc140280 0x7f68cc0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d4733640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 0x7f68cc0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d4733640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0af9f0 con 0x7f68cc140280 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d4733640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc14f900 con 0x7f68cc06aa00 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d4733640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0af710 con 0x7f68cc15d8c0 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d24cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 0x7f68cc14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d2cd0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 0x7f68cc0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d24cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 0x7f68cc14f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38962/0 (socket says 172.21.15.192:38962) 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d24cf640 1 -- 172.21.15.192:0/3743020546 learned_addr learned my addr 172.21.15.192:0/3743020546 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.314+0000 7f68d2cd0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 0x7f68cc0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:45996/0 (socket says 172.21.15.192:45996) 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d1cce640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68cc140280 0x7f68cc0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d24cf640 1 -- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 msgr2=0x7f68cc0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d24cf640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 0x7f68cc0b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d24cf640 1 -- 172.21.15.192:0/3743020546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68cc140280 msgr2=0x7f68cc0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d24cf640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68cc140280 0x7f68cc0ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d24cf640 1 -- 172.21.15.192:0/3743020546 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009a70 con 0x7f68cc06aa00 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d2cd0640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 0x7f68cc0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d24cf640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 0x7f68cc14f3c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00c4d0 tx=0x7f68bc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0052d0 con 0x7f68cc06aa00 2023-12-04T05:38:53.320 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc005430 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80094d0 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc003810 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc0ad9e0 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f68bc003c70 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.315+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f68bc01c440 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.316+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc067460 con 0x7f68cc06aa00 2023-12-04T05:38:53.321 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.318+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f68bc017070 con 0x7f68cc06aa00 2023-12-04T05:38:53.391 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.390+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5c1fd699-00ab-4e49-bfd2-d2d3b9406819", "id": 1} v 0) v1 -- 0x7f68cc06eab0 con 0x7f68cc06aa00 2023-12-04T05:38:53.399 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.398+0000 7f68c2ffd640 1 -- 172.21.15.192:0/3743020546 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c1fd699-00ab-4e49-bfd2-d2d3b9406819", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f68cc06ae50 con 0x7f68cc06aa00 2023-12-04T05:38:53.399 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-04T05:38:53.400 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 msgr2=0x7f68cc14f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.400 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 0x7f68cc14f3c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00c4d0 tx=0x7f68bc00c8d0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 shutdown_connections 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc15d8c0 0x7f68cc0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68cc140280 0x7f68cc0ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 --2- 172.21.15.192:0/3743020546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc06aa00 0x7f68cc14f3c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 >> 172.21.15.192:0/3743020546 conn(0x7f68cc0b1de0 msgr2=0x7f68cc0b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 shutdown_connections 2023-12-04T05:38:53.401 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.399+0000 7f68d4733640 1 -- 172.21.15.192:0/3743020546 wait complete. 2023-12-04T05:38:53.411 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new f801a2ee-b5d9-4d07-990b-2a19c10b639e 2 2023-12-04T05:38:53.473 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.472+0000 7f868bc39040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-04T05:38:53.499 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.498+0000 7f14d0213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-04T05:38:53.511 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.510+0000 7f408f5a5640 1 Processor -- start 2023-12-04T05:38:53.512 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408f5a5640 1 -- start start 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408f5a5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4088144390 0x7f40880d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408f5a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4088070980 0x7f40880cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408f5a5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40880cdf40 con 0x7f408806fe00 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408f5a5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40880d0ef0 con 0x7f4088070980 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408f5a5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40880ce220 con 0x7f4088144390 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408d341640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4088144390 0x7f40880d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.513 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408d341640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4088144390 0x7f40880d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38974/0 (socket says 172.21.15.192:38974) 2023-12-04T05:38:53.514 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.511+0000 7f408d341640 1 -- 172.21.15.192:0/3331975523 learned_addr learned my addr 172.21.15.192:0/3331975523 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.514 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.512+0000 7f408db42640 1 --2- 172.21.15.192:0/3331975523 >> v2:172.21.15.111:3300/0 conn(0x7f4088070980 0x7f40880cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.514 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.512+0000 7f408d341640 1 -- 172.21.15.192:0/3331975523 >> v2:172.21.15.111:3300/0 conn(0x7f4088070980 msgr2=0x7f40880cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:53.514 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.512+0000 7f408d341640 1 --2- 172.21.15.192:0/3331975523 >> v2:172.21.15.111:3300/0 conn(0x7f4088070980 0x7f40880cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.514 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.512+0000 7f408d341640 1 -- 172.21.15.192:0/3331975523 >> v1:172.21.15.192:6789/0 conn(0x7f408806fe00 legacy=0x7f4088070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.514 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.512+0000 7f408d341640 1 -- 172.21.15.192:0/3331975523 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40880cb810 con 0x7f4088144390 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408d341640 1 --2- 172.21.15.192:0/3331975523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40880d0810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f407c009870 tx=0x7f407c02ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a983d79577708cc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f407bfff640 1 -- 172.21.15.192:0/3331975523 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c02f840 con 0x7f4088144390 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f407bfff640 1 -- 172.21.15.192:0/3331975523 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c02f9a0 con 0x7f4088144390 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f407bfff640 1 -- 172.21.15.192:0/3331975523 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c02fd30 con 0x7f4088144390 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 -- 172.21.15.192:0/3331975523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 msgr2=0x7f40880d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 --2- 172.21.15.192:0/3331975523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40880d0810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f407c009870 tx=0x7f407c02ba20 comp rx=0 tx=0).stop 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 -- 172.21.15.192:0/3331975523 shutdown_connections 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 --2- 172.21.15.192:0/3331975523 >> v2:172.21.15.111:3300/0 conn(0x7f4088070980 0x7f40880cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.516 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 --2- 172.21.15.192:0/3331975523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40880d0810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.517 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 -- 172.21.15.192:0/3331975523 >> 172.21.15.192:0/3331975523 conn(0x7f408806e7a0 msgr2=0x7f408806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.517 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 -- 172.21.15.192:0/3331975523 shutdown_connections 2023-12-04T05:38:53.517 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.513+0000 7f408f5a5640 1 -- 172.21.15.192:0/3331975523 wait complete. 2023-12-04T05:38:53.517 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 Processor -- start 2023-12-04T05:38:53.518 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 -- start start 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408806fe00 0x7f408813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 0x7f40881f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40881f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40881f4540 con 0x7f408806fe00 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408815ac20 con 0x7f4088144390 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408f5a5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408813de40 con 0x7f4088070980 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408cb40640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 0x7f40881f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408db42640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40881f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408cb40640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 0x7f40881f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46004/0 (socket says 172.21.15.192:46004) 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.515+0000 7f408cb40640 1 -- 172.21.15.192:0/2343266944 learned_addr learned my addr 172.21.15.192:0/2343266944 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408d341640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408806fe00 0x7f408813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408cb40640 1 -- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 msgr2=0x7f40881f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.520 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408cb40640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40881f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408cb40640 1 -- 172.21.15.192:0/2343266944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408806fe00 msgr2=0x7f408813d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408cb40640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408806fe00 0x7f408813d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408cb40640 1 -- 172.21.15.192:0/2343266944 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0094d0 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408db42640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40881f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408d341640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408806fe00 0x7f408813d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408cb40640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 0x7f40881f2d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f408400c450 tx=0x7f408400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840052d0 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084005430 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40881f3930 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084012870 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40881f3ef0 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.516+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4084012cd0 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.517+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4084003a10 con 0x7f4088070980 2023-12-04T05:38:53.521 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.517+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40880c5f60 con 0x7f4088070980 2023-12-04T05:38:53.522 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.519+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4084003bd0 con 0x7f4088070980 2023-12-04T05:38:53.525 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.524+0000 7f868bc39040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-04T05:38:53.555 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.554+0000 7f14d0213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-04T05:38:53.569 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.567+0000 7f868bc39040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-04T05:38:53.592 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.591+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f801a2ee-b5d9-4d07-990b-2a19c10b639e", "id": 2} v 0) v1 -- 0x7f4088067460 con 0x7f4088070980 2023-12-04T05:38:53.597 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.596+0000 7f4079ffb640 1 -- 172.21.15.192:0/2343266944 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f801a2ee-b5d9-4d07-990b-2a19c10b639e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f40840036c0 con 0x7f4088070980 2023-12-04T05:38:53.598 INFO:teuthology.orchestra.run.smithi192.stdout:2 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.597+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 msgr2=0x7f40881f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.597+0000 7f408f5a5640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 0x7f40881f2d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f408400c450 tx=0x7f408400c910 comp rx=0 tx=0).stop 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 shutdown_connections 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4088144390 0x7f40881f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4088070980 0x7f40881f2d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 --2- 172.21.15.192:0/2343266944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f408806fe00 0x7f408813d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.599 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 >> 172.21.15.192:0/2343266944 conn(0x7f408806e7a0 msgr2=0x7f40880b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.600 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 shutdown_connections 2023-12-04T05:38:53.600 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.598+0000 7f408f5a5640 1 -- 172.21.15.192:0/2343266944 wait complete. 2023-12-04T05:38:53.601 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.601+0000 7f14d0213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-04T05:38:53.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 6ce755a0-a57a-43e4-ae57-a640139bc901 3 2023-12-04T05:38:53.621 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.620+0000 7f868bc39040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-04T05:38:53.659 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.658+0000 7f14d0213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-04T05:38:53.663 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.662+0000 7f868bc39040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-04T05:38:53.703 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.702+0000 7f14d0213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-04T05:38:53.710 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.707+0000 7fa58523d640 1 Processor -- start 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.707+0000 7fa58523d640 1 -- start start 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa58523d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa5800d0a60 0x7fa5800ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa58523d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58015b990 con 0x7fa58006eed0 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa58523d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58015bc70 con 0x7fa5800ccef0 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa58523d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58015bf50 con 0x7fa5800d0a60 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa57ed9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa5800d0a60 0x7fa5800ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa57ed9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa5800d0a60 0x7fa5800ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38988/0 (socket says 172.21.15.192:38988) 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa57ed9d640 1 -- 172.21.15.192:0/1935029678 learned_addr learned my addr 172.21.15.192:0/1935029678 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa57ed9d640 1 -- 172.21.15.192:0/1935029678 >> v1:172.21.15.192:6789/0 conn(0x7fa5800ccef0 legacy=0x7fa58006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa57ed9d640 1 -- 172.21.15.192:0/1935029678 >> v1:172.21.15.111:6789/0 conn(0x7fa58006eed0 legacy=0x7fa58015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.708+0000 7fa57ed9d640 1 -- 172.21.15.192:0/1935029678 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58015c230 con 0x7fa5800d0a60 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.709+0000 7fa57ed9d640 1 --2- 172.21.15.192:0/1935029678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5800d0a60 0x7fa5800ceb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa574009870 tx=0x7fa57402ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3b0a6fab696578 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.711 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.709+0000 7fa57dd9b640 1 -- 172.21.15.192:0/1935029678 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57402f840 con 0x7fa5800d0a60 2023-12-04T05:38:53.712 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.709+0000 7fa57dd9b640 1 -- 172.21.15.192:0/1935029678 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57402f9a0 con 0x7fa5800d0a60 2023-12-04T05:38:53.712 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.709+0000 7fa57dd9b640 1 -- 172.21.15.192:0/1935029678 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57402fd30 con 0x7fa5800d0a60 2023-12-04T05:38:53.713 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 -- 172.21.15.192:0/1935029678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5800d0a60 msgr2=0x7fa5800ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.713 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 --2- 172.21.15.192:0/1935029678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5800d0a60 0x7fa5800ceb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa574009870 tx=0x7fa57402ba20 comp rx=0 tx=0).stop 2023-12-04T05:38:53.713 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 -- 172.21.15.192:0/1935029678 shutdown_connections 2023-12-04T05:38:53.713 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 --2- 172.21.15.192:0/1935029678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5800d0a60 0x7fa5800ceb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.713 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 -- 172.21.15.192:0/1935029678 >> 172.21.15.192:0/1935029678 conn(0x7fa5800d0f40 msgr2=0x7fa5800d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.714 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 -- 172.21.15.192:0/1935029678 shutdown_connections 2023-12-04T05:38:53.714 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.710+0000 7fa58523d640 1 -- 172.21.15.192:0/1935029678 wait complete. 2023-12-04T05:38:53.714 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.711+0000 7fa58523d640 1 Processor -- start 2023-12-04T05:38:53.714 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.711+0000 7fa58523d640 1 -- start start 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa58523d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 0x7fa5800b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa58523d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5800ccef0 0x7fa58014e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa58523d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 0x7fa5800ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa58523d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800b4a30 con 0x7fa5800ccef0 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa58523d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800b5f20 con 0x7fa58006eed0 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa58523d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800b5c40 con 0x7fa58014e9c0 2023-12-04T05:38:53.716 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa57ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 0x7fa5800b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa57f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 0x7fa5800ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa57f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 0x7fa5800ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46010/0 (socket says 172.21.15.192:46010) 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa57ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 0x7fa5800b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39004/0 (socket says 172.21.15.192:39004) 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa57f59e640 1 -- 172.21.15.192:0/3701546466 learned_addr learned my addr 172.21.15.192:0/3701546466 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.712+0000 7fa57e59c640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5800ccef0 0x7fa58014e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57ed9d640 1 -- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 msgr2=0x7fa5800ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57ed9d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 0x7fa5800ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57ed9d640 1 -- 172.21.15.192:0/3701546466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5800ccef0 msgr2=0x7fa58014e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57ed9d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5800ccef0 0x7fa58014e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57ed9d640 1 -- 172.21.15.192:0/3701546466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740094d0 con 0x7fa58006eed0 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57f59e640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 0x7fa5800ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa57ed9d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 0x7fa5800b5410 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa574033000 tx=0x7fa574004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.717 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574045070 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57402f810 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5801f6a20 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574041400 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5801f6fe0 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa574041860 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.713+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa57403c070 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.714+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa580067460 con 0x7fa58006eed0 2023-12-04T05:38:53.718 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.716+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa574041ca0 con 0x7fa58006eed0 2023-12-04T05:38:53.763 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.762+0000 7f868bc39040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-04T05:38:53.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.788+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ce755a0-a57a-43e4-ae57-a640139bc901", "id": 3} v 0) v1 -- 0x7fa5800cb320 con 0x7fa58006eed0 2023-12-04T05:38:53.794 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.793+0000 7fa5677fe640 1 -- 172.21.15.192:0/3701546466 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ce755a0-a57a-43e4-ae57-a640139bc901", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5740580b0 con 0x7fa58006eed0 2023-12-04T05:38:53.794 INFO:teuthology.orchestra.run.smithi192.stdout:3 2023-12-04T05:38:53.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 msgr2=0x7fa5800b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.795 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 0x7fa5800b5410 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa574033000 tx=0x7fa574004730 comp rx=0 tx=0).stop 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 shutdown_connections 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa58014e9c0 0x7fa5800ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5800ccef0 0x7fa58014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 --2- 172.21.15.192:0/3701546466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa58006eed0 0x7fa5800b5410 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 >> 172.21.15.192:0/3701546466 conn(0x7fa5800d0f40 msgr2=0x7fa5800b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 shutdown_connections 2023-12-04T05:38:53.796 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.794+0000 7fa58523d640 1 -- 172.21.15.192:0/3701546466 wait complete. 2023-12-04T05:38:53.806 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new ab580f43-28c3-4b07-85a3-9140109388c9 4 2023-12-04T05:38:53.809 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.809+0000 7f14d0213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-04T05:38:53.833 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.832+0000 7f868bc39040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-04T05:38:53.878 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.877+0000 7f868bc39040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-04T05:38:53.884 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.883+0000 7f14d0213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-04T05:38:53.905 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.904+0000 7f7d49e44640 1 Processor -- start 2023-12-04T05:38:53.906 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.904+0000 7f7d49e44640 1 -- start start 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d49e44640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7d44157a20 0x7f7d4406a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d49e44640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d49e44640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d440ad640 con 0x7f7d4406aa00 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d49e44640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d44157e00 con 0x7f7d44157a20 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d49e44640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4406af80 con 0x7f7d440b4a30 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d43fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d43fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46016/0 (socket says 172.21.15.192:46016) 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d43fff640 1 -- 172.21.15.192:0/2011214813 learned_addr learned my addr 172.21.15.192:0/2011214813 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.905+0000 7f7d437fe640 1 --2- 172.21.15.192:0/2011214813 >> v2:172.21.15.111:3300/0 conn(0x7f7d44157a20 0x7f7d4406a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.908 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.906+0000 7f7d43fff640 1 -- 172.21.15.192:0/2011214813 >> v2:172.21.15.111:3300/0 conn(0x7f7d44157a20 msgr2=0x7f7d4406a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:53.908 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.906+0000 7f7d43fff640 1 --2- 172.21.15.192:0/2011214813 >> v2:172.21.15.111:3300/0 conn(0x7f7d44157a20 0x7f7d4406a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.908 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.906+0000 7f7d43fff640 1 -- 172.21.15.192:0/2011214813 >> v1:172.21.15.111:6789/0 conn(0x7f7d4406aa00 legacy=0x7f7d440b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.908 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.906+0000 7f7d43fff640 1 -- 172.21.15.192:0/2011214813 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4415d990 con 0x7f7d440b4a30 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.907+0000 7f7d43fff640 1 --2- 172.21.15.192:0/2011214813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44160e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009e50 tx=0x7f7d3c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991bc70f78e6e193 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.907+0000 7f7d427fc640 1 -- 172.21.15.192:0/2011214813 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c02cb00 con 0x7f7d440b4a30 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.907+0000 7f7d427fc640 1 -- 172.21.15.192:0/2011214813 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c02cc60 con 0x7f7d440b4a30 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.907+0000 7f7d427fc640 1 -- 172.21.15.192:0/2011214813 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0343f0 con 0x7f7d440b4a30 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 -- 172.21.15.192:0/2011214813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 msgr2=0x7f7d44160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 --2- 172.21.15.192:0/2011214813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44160e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009e50 tx=0x7f7d3c02bf60 comp rx=0 tx=0).stop 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 -- 172.21.15.192:0/2011214813 shutdown_connections 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 --2- 172.21.15.192:0/2011214813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44160e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 --2- 172.21.15.192:0/2011214813 >> v2:172.21.15.111:3300/0 conn(0x7f7d44157a20 0x7f7d4406a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 -- 172.21.15.192:0/2011214813 >> 172.21.15.192:0/2011214813 conn(0x7f7d4406d4a0 msgr2=0x7f7d4406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 -- 172.21.15.192:0/2011214813 shutdown_connections 2023-12-04T05:38:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.908+0000 7f7d49e44640 1 -- 172.21.15.192:0/2011214813 wait complete. 2023-12-04T05:38:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.909+0000 7f7d49e44640 1 Processor -- start 2023-12-04T05:38:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.909+0000 7f7d49e44640 1 -- start start 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 0x7f7d4415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d44157a20 0x7f7d44158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440cf680 con 0x7f7d44157a20 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44159760 con 0x7f7d4406aa00 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44158b50 con 0x7f7d440b4a30 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 0x7f7d4415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 0x7f7d4415a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39014/0 (socket says 172.21.15.192:39014) 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 -- 172.21.15.192:0/958225828 learned_addr learned my addr 172.21.15.192:0/958225828 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d43fff640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d44157a20 0x7f7d44158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 -- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 msgr2=0x7f7d44157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.913 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 -- 172.21.15.192:0/958225828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d44157a20 msgr2=0x7f7d44158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d44157a20 0x7f7d44158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 -- 172.21.15.192:0/958225828 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c009b30 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d42ffd640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d437fe640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 0x7f7d4415a380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800e930 tx=0x7f7d3800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38009690 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.910+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d44158e30 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.911+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800ca00 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.911+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800cdb0 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.911+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d441593f0 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.911+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7d38005c90 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.911+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7d380139d0 con 0x7f7d4406aa00 2023-12-04T05:38:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.911+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44067460 con 0x7f7d4406aa00 2023-12-04T05:38:53.915 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.914+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7d38021060 con 0x7f7d4406aa00 2023-12-04T05:38:53.930 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.929+0000 7f868bc39040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-04T05:38:53.933 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.932+0000 7f14d0213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-04T05:38:53.976 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:53.975+0000 7f868bc39040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-04T05:38:53.986 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.985+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ab580f43-28c3-4b07-85a3-9140109388c9", "id": 4} v 0) v1 -- 0x7f7d440afc00 con 0x7f7d4406aa00 2023-12-04T05:38:53.987 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:53.986+0000 7f14d0213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-04T05:38:53.992 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d2bfff640 1 -- 172.21.15.192:0/958225828 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab580f43-28c3-4b07-85a3-9140109388c9", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d38017070 con 0x7f7d4406aa00 2023-12-04T05:38:53.992 INFO:teuthology.orchestra.run.smithi192.stdout:4 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 msgr2=0x7f7d4415a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d49e44640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 0x7f7d4415a380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800e930 tx=0x7f7d3800edf0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 shutdown_connections 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d49e44640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d44157a20 0x7f7d44158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d49e44640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d440b4a30 0x7f7d44157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.991+0000 7f7d49e44640 1 --2- 172.21.15.192:0/958225828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d4406aa00 0x7f7d4415a380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.992+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 >> 172.21.15.192:0/958225828 conn(0x7f7d4406d4a0 msgr2=0x7f7d440b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:53.993 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.992+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 shutdown_connections 2023-12-04T05:38:53.994 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:53.992+0000 7f7d49e44640 1 -- 172.21.15.192:0/958225828 wait complete. 2023-12-04T05:38:54.003 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 70358cad-93c5-4a17-ac66-6a33bf19c0ec 5 2023-12-04T05:38:54.036 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.035+0000 7f14d0213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-04T05:38:54.136 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.135+0000 7f79dcecf640 1 Processor -- start 2023-12-04T05:38:54.136 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.135+0000 7f79dcecf640 1 -- start start 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79dcecf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f79d80b4320 0x7f79d80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79dcecf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f79d8160e70 0x7f79d8161300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79dcecf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80ad640 con 0x7f79d8157a20 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79dcecf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d8157e00 con 0x7f79d80b4320 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79dcecf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80b4ce0 con 0x7f79d8160e70 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f79d8160e70 0x7f79d8161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f79d8160e70 0x7f79d8161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39030/0 (socket says 172.21.15.192:39030) 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 -- 172.21.15.192:0/2633887246 learned_addr learned my addr 172.21.15.192:0/2633887246 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d5d9b640 1 --2- 172.21.15.192:0/2633887246 >> v2:172.21.15.111:3300/0 conn(0x7f79d80b4320 0x7f79d80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 -- 172.21.15.192:0/2633887246 >> v2:172.21.15.111:3300/0 conn(0x7f79d80b4320 msgr2=0x7f79d80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:54.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 --2- 172.21.15.192:0/2633887246 >> v2:172.21.15.111:3300/0 conn(0x7f79d80b4320 0x7f79d80b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 -- 172.21.15.192:0/2633887246 >> v1:172.21.15.111:6789/0 conn(0x7f79d8157a20 legacy=0x7f79d806a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.136+0000 7f79d6d9d640 1 -- 172.21.15.192:0/2633887246 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d806af80 con 0x7f79d8160e70 2023-12-04T05:38:54.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.137+0000 7f79d6d9d640 1 --2- 172.21.15.192:0/2633887246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d8161300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79d80cc250 tx=0x7f79d002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9779a233872ba44e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.137+0000 7f79d559a640 1 -- 172.21.15.192:0/2633887246 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d002cb20 con 0x7f79d8160e70 2023-12-04T05:38:54.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.137+0000 7f79d559a640 1 -- 172.21.15.192:0/2633887246 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d002cc80 con 0x7f79d8160e70 2023-12-04T05:38:54.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.137+0000 7f79d559a640 1 -- 172.21.15.192:0/2633887246 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00343f0 con 0x7f79d8160e70 2023-12-04T05:38:54.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 -- 172.21.15.192:0/2633887246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 msgr2=0x7f79d8161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 --2- 172.21.15.192:0/2633887246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d8161300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79d80cc250 tx=0x7f79d002bf80 comp rx=0 tx=0).stop 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 -- 172.21.15.192:0/2633887246 shutdown_connections 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 --2- 172.21.15.192:0/2633887246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d8161300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 --2- 172.21.15.192:0/2633887246 >> v2:172.21.15.111:3300/0 conn(0x7f79d80b4320 0x7f79d80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 -- 172.21.15.192:0/2633887246 >> 172.21.15.192:0/2633887246 conn(0x7f79d806d4a0 msgr2=0x7f79d806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 -- 172.21.15.192:0/2633887246 shutdown_connections 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.138+0000 7f79dcecf640 1 -- 172.21.15.192:0/2633887246 wait complete. 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.139+0000 7f79dcecf640 1 Processor -- start 2023-12-04T05:38:54.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.139+0000 7f79dcecf640 1 -- start start 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79dcecf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 0x7f79d81f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79dcecf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8157a20 0x7f79d81556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79dcecf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79dcecf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80ce4a0 con 0x7f79d8157a20 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79dcecf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cf880 con 0x7f79d8160e70 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79dcecf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d81f34d0 con 0x7f79d80b4320 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d659c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 0x7f79d81f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d659c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 0x7f79d81f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46034/0 (socket says 172.21.15.192:46034) 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d80cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39036/0 (socket says 172.21.15.192:39036) 2023-12-04T05:38:54.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d659c640 1 -- 172.21.15.192:0/3924655646 learned_addr learned my addr 172.21.15.192:0/3924655646 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d5d9b640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8157a20 0x7f79d81556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 -- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 msgr2=0x7f79d81f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 0x7f79d81f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 -- 172.21.15.192:0/3924655646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8157a20 msgr2=0x7f79d81556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8157a20 0x7f79d81556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 -- 172.21.15.192:0/3924655646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009b30 con 0x7f79d8160e70 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d659c640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 0x7f79d81f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d5d9b640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8157a20 0x7f79d81556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.140+0000 7f79d6d9d640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d80cd820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f79d0032000 tx=0x7f79d0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0009510 con 0x7f79d8160e70 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0007e00 con 0x7f79d8160e70 2023-12-04T05:38:54.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8159c90 con 0x7f79d8160e70 2023-12-04T05:38:54.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d003e690 con 0x7f79d8160e70 2023-12-04T05:38:54.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d8159920 con 0x7f79d8160e70 2023-12-04T05:38:54.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f79d003eaf0 con 0x7f79d8160e70 2023-12-04T05:38:54.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.141+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f79d0048de0 con 0x7f79d8160e70 2023-12-04T05:38:54.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.142+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8159e20 con 0x7f79d8160e70 2023-12-04T05:38:54.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.144+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f79d0066ce0 con 0x7f79d8160e70 2023-12-04T05:38:54.217 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.216+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "70358cad-93c5-4a17-ac66-6a33bf19c0ec", "id": 5} v 0) v1 -- 0x7f79d8159470 con 0x7f79d8160e70 2023-12-04T05:38:54.223 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.222+0000 7f79c6ffd640 1 -- 172.21.15.192:0/3924655646 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70358cad-93c5-4a17-ac66-6a33bf19c0ec", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f79d0055030 con 0x7f79d8160e70 2023-12-04T05:38:54.223 INFO:teuthology.orchestra.run.smithi192.stdout:5 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.222+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 msgr2=0x7f79d80cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.222+0000 7f79dcecf640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d80cd820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f79d0032000 tx=0x7f79d0004a40 comp rx=0 tx=0).stop 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 shutdown_connections 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8160e70 0x7f79d80cd820 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8157a20 0x7f79d81556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 --2- 172.21.15.192:0/3924655646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80b4320 0x7f79d81f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 >> 172.21.15.192:0/3924655646 conn(0x7f79d806d4a0 msgr2=0x7f79d8071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.224 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 shutdown_connections 2023-12-04T05:38:54.225 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.223+0000 7f79dcecf640 1 -- 172.21.15.192:0/3924655646 wait complete. 2023-12-04T05:38:54.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new db4a5ec8-5bf0-4980-8569-32f1e4e11d2e 6 2023-12-04T05:38:54.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 Processor -- start 2023-12-04T05:38:54.335 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 -- start start 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c00b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f10c006abd0 0x7f10c00ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00b38d0 con 0x7f10c00b4670 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00af8a0 con 0x7f10c006abd0 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.334+0000 7f10c4b68640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00b4ca0 con 0x7f10c015d4a0 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c00b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c00b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46038/0 (socket says 172.21.15.192:46038) 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 -- 172.21.15.192:0/4257469068 learned_addr learned my addr 172.21.15.192:0/4257469068 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10bed9d640 1 --2- 172.21.15.192:0/4257469068 >> v2:172.21.15.111:3300/0 conn(0x7f10c006abd0 0x7f10c00ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 -- 172.21.15.192:0/4257469068 >> v2:172.21.15.111:3300/0 conn(0x7f10c006abd0 msgr2=0x7f10c00ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:54.337 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 --2- 172.21.15.192:0/4257469068 >> v2:172.21.15.111:3300/0 conn(0x7f10c006abd0 0x7f10c00ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.338 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 -- 172.21.15.192:0/4257469068 >> v1:172.21.15.111:6789/0 conn(0x7f10c00b4670 legacy=0x7f10c006a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.338 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.335+0000 7f10be59c640 1 -- 172.21.15.192:0/4257469068 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c00b2fa0 con 0x7f10c015d4a0 2023-12-04T05:38:54.338 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10be59c640 1 --2- 172.21.15.192:0/4257469068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c00b4130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009870 tx=0x7f10b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775d06ab1b4f311c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.338 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10bdd9b640 1 -- 172.21.15.192:0/4257469068 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0030840 con 0x7f10c015d4a0 2023-12-04T05:38:54.338 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10bdd9b640 1 -- 172.21.15.192:0/4257469068 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b00309a0 con 0x7f10c015d4a0 2023-12-04T05:38:54.338 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10bdd9b640 1 -- 172.21.15.192:0/4257469068 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0030d30 con 0x7f10c015d4a0 2023-12-04T05:38:54.341 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 -- 172.21.15.192:0/4257469068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 msgr2=0x7f10c00b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.341 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 --2- 172.21.15.192:0/4257469068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c00b4130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009870 tx=0x7f10b002b980 comp rx=0 tx=0).stop 2023-12-04T05:38:54.342 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 -- 172.21.15.192:0/4257469068 shutdown_connections 2023-12-04T05:38:54.342 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 --2- 172.21.15.192:0/4257469068 >> v2:172.21.15.111:3300/0 conn(0x7f10c006abd0 0x7f10c00ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.342 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 --2- 172.21.15.192:0/4257469068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c00b4130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.342 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 -- 172.21.15.192:0/4257469068 >> 172.21.15.192:0/4257469068 conn(0x7f10c00b1c00 msgr2=0x7f10c00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.342 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 -- 172.21.15.192:0/4257469068 shutdown_connections 2023-12-04T05:38:54.342 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.336+0000 7f10c4b68640 1 -- 172.21.15.192:0/4257469068 wait complete. 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 Processor -- start 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 -- start start 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10c006abd0 0x7f10c014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10c00b4670 0x7f10c015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00ce150 con 0x7f10c006abd0 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c014bc80 con 0x7f10c00b4670 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.338+0000 7f10c4b68640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c0158650 con 0x7f10c015d4a0 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10c00b4670 0x7f10c015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c015c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46044/0 (socket says 172.21.15.192:46044) 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 -- 172.21.15.192:0/1753818227 learned_addr learned my addr 172.21.15.192:0/1753818227 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10b7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10c00b4670 0x7f10c015bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39052/0 (socket says 172.21.15.192:39052) 2023-12-04T05:38:54.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10be59c640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10c006abd0 0x7f10c014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 -- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10c00b4670 msgr2=0x7f10c015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10c00b4670 0x7f10c015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 -- 172.21.15.192:0/1753818227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10c006abd0 msgr2=0x7f10c014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10c006abd0 0x7f10c014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 -- 172.21.15.192:0/1753818227 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00094d0 con 0x7f10c015d4a0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10be59c640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10c006abd0 0x7f10c014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10bed9d640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c015c270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10b800efc0 tx=0x7f10b800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8014480 con 0x7f10c015d4a0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b800f040 con 0x7f10c015d4a0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8014820 con 0x7f10c015d4a0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c00b3ee0 con 0x7f10c015d4a0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c00cda90 con 0x7f10c015d4a0 2023-12-04T05:38:54.344 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.339+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f10b8014c80 con 0x7f10c015d4a0 2023-12-04T05:38:54.345 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.340+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f10b801dcc0 con 0x7f10c015d4a0 2023-12-04T05:38:54.345 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.340+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c00b3d40 con 0x7f10c015d4a0 2023-12-04T05:38:54.345 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.342+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f10b803bce0 con 0x7f10c015d4a0 2023-12-04T05:38:54.351 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.350+0000 7f868bc39040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-04T05:38:54.414 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.413+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "db4a5ec8-5bf0-4980-8569-32f1e4e11d2e", "id": 6} v 0) v1 -- 0x7f10c00ce290 con 0x7f10c015d4a0 2023-12-04T05:38:54.419 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.418+0000 7f10b6ffd640 1 -- 172.21.15.192:0/1753818227 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db4a5ec8-5bf0-4980-8569-32f1e4e11d2e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f10c00ce290 con 0x7f10c015d4a0 2023-12-04T05:38:54.420 INFO:teuthology.orchestra.run.smithi192.stdout:6 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 msgr2=0x7f10c015c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c015c270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10b800efc0 tx=0x7f10b800c490 comp rx=0 tx=0).stop 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 shutdown_connections 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10c015d4a0 0x7f10c015c270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10c00b4670 0x7f10c015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 --2- 172.21.15.192:0/1753818227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10c006abd0 0x7f10c014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 >> 172.21.15.192:0/1753818227 conn(0x7f10c00b1c00 msgr2=0x7f10c00af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 shutdown_connections 2023-12-04T05:38:54.421 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.419+0000 7f10c4b68640 1 -- 172.21.15.192:0/1753818227 wait complete. 2023-12-04T05:38:54.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2 7 2023-12-04T05:38:54.436 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.436+0000 7f14d0213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-04T05:38:54.486 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.485+0000 7f868bc39040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-04T05:38:54.531 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.530+0000 7f2fb009b640 1 Processor -- start 2023-12-04T05:38:54.531 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.530+0000 7f2fb009b640 1 -- start start 2023-12-04T05:38:54.532 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fb009b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa8157a20 0x7f2fa806a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.532 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fb009b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa80ad640 con 0x7f2fa806aa00 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fb009b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa806af80 con 0x7f2fa80b4a30 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fb009b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8157e00 con 0x7f2fa8157a20 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fade37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa8157a20 0x7f2fa806a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fae638640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f2fa80b4a30 0x7f2fa8160e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:56410/0 (socket says 172.21.15.192:56410) 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fae638640 1 -- 172.21.15.192:0/217198919 learned_addr learned my addr 172.21.15.192:0/217198919 (peer_addr_for_me v1:172.21.15.192:0/0) 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2face35640 1 -- 172.21.15.192:0/217198919 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460379107 0 0) 0x7f2fa806af80 con 0x7f2fa80b4a30 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2face35640 1 -- 172.21.15.192:0/217198919 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f94003520 con 0x7f2fa80b4a30 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fade37640 1 -- 172.21.15.192:0/217198919 >> v1:172.21.15.192:6789/0 conn(0x7f2fa80b4a30 legacy=0x7f2fa8160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fade37640 1 -- 172.21.15.192:0/217198919 >> v1:172.21.15.111:6789/0 conn(0x7f2fa806aa00 legacy=0x7f2fa80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.533 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.531+0000 7f2fade37640 1 -- 172.21.15.192:0/217198919 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa815d990 con 0x7f2fa8157a20 2023-12-04T05:38:54.534 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.532+0000 7f2fade37640 1 --2- 172.21.15.192:0/217198919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa8157a20 0x7f2fa806a4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40097d0 tx=0x7f2fa402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777eff9b87843844 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.534 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.532+0000 7f2face35640 1 -- 172.21.15.192:0/217198919 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031890 con 0x7f2fa8157a20 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.532+0000 7f2face35640 1 -- 172.21.15.192:0/217198919 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40319f0 con 0x7f2fa8157a20 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.532+0000 7f2face35640 1 -- 172.21.15.192:0/217198919 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031d80 con 0x7f2fa8157a20 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 -- 172.21.15.192:0/217198919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa8157a20 msgr2=0x7f2fa806a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 --2- 172.21.15.192:0/217198919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa8157a20 0x7f2fa806a4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40097d0 tx=0x7f2fa402b930 comp rx=0 tx=0).stop 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 -- 172.21.15.192:0/217198919 shutdown_connections 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 --2- 172.21.15.192:0/217198919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa8157a20 0x7f2fa806a4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 -- 172.21.15.192:0/217198919 >> 172.21.15.192:0/217198919 conn(0x7f2fa806d4a0 msgr2=0x7f2fa806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 -- 172.21.15.192:0/217198919 shutdown_connections 2023-12-04T05:38:54.535 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.533+0000 7f2fb009b640 1 -- 172.21.15.192:0/217198919 wait complete. 2023-12-04T05:38:54.538 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.534+0000 7f2fb009b640 1 Processor -- start 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.534+0000 7f2fb009b640 1 -- start start 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fa806aa00 0x7f2fa815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 0x7f2fa8159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 0x7f2fa8142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa815c870 con 0x7f2fa806aa00 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa815b9e0 con 0x7f2fa8157a20 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8159ec0 con 0x7f2fa80b4a30 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 0x7f2fa8142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fad636640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 0x7f2fa8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 0x7f2fa8142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39056/0 (socket says 172.21.15.192:39056) 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 -- 172.21.15.192:0/1369613514 learned_addr learned my addr 172.21.15.192:0/1369613514 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fad636640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 0x7f2fa8159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:46064/0 (socket says 172.21.15.192:46064) 2023-12-04T05:38:54.539 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fade37640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fa806aa00 0x7f2fa815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 -- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 msgr2=0x7f2fa8159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 0x7f2fa8159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 -- 172.21.15.192:0/1369613514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fa806aa00 msgr2=0x7f2fa815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fa806aa00 0x7f2fa815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 -- 172.21.15.192:0/1369613514 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4009400 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fad636640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 0x7f2fa8159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fae638640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 0x7f2fa8142860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000caa0 tx=0x7f2fa000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000ac00 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa000ad60 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa815bbf0 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.535+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0012620 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.536+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa815c1b0 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.536+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2fa0012a80 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.536+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f2fa00076d0 con 0x7f2fa8157a20 2023-12-04T05:38:54.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.537+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa815c9b0 con 0x7f2fa8157a20 2023-12-04T05:38:54.541 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.539+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2fa0017070 con 0x7f2fa8157a20 2023-12-04T05:38:54.579 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.578+0000 7f14d0213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-04T05:38:54.593 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.592+0000 7f868bc39040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-04T05:38:54.612 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.610+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2", "id": 7} v 0) v1 -- 0x7f2fa8067460 con 0x7f2fa8157a20 2023-12-04T05:38:54.617 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.616+0000 7f2f9e7fc640 1 -- 172.21.15.192:0/1369613514 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2fa0007920 con 0x7f2fa8157a20 2023-12-04T05:38:54.618 INFO:teuthology.orchestra.run.smithi192.stdout:7 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 msgr2=0x7f2fa8142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 0x7f2fa8142860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000caa0 tx=0x7f2fa000cf60 comp rx=0 tx=0).stop 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 shutdown_connections 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fa8157a20 0x7f2fa8142860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2fa80b4a30 0x7f2fa8159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 --2- 172.21.15.192:0/1369613514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fa806aa00 0x7f2fa815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 >> 172.21.15.192:0/1369613514 conn(0x7f2fa806d4a0 msgr2=0x7f2fa80b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 shutdown_connections 2023-12-04T05:38:54.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:38:54.617+0000 7f2fb009b640 1 -- 172.21.15.192:0/1369613514 wait complete. 2023-12-04T05:38:54.629 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-04T05:38:54.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-04T05:38:54.631 INFO:tasks.ceph.osd.0:Started 2023-12-04T05:38:54.631 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-04T05:38:54.631 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-04T05:38:54.633 INFO:tasks.ceph.osd.1:Started 2023-12-04T05:38:54.633 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-04T05:38:54.633 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-04T05:38:54.636 INFO:tasks.ceph.osd.2:Started 2023-12-04T05:38:54.636 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-04T05:38:54.636 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-04T05:38:54.639 INFO:tasks.ceph.osd.3:Started 2023-12-04T05:38:54.640 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-04T05:38:54.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-04T05:38:54.640 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.639+0000 7f868bc39040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-04T05:38:54.671 INFO:tasks.ceph.osd.4:Started 2023-12-04T05:38:54.671 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-04T05:38:54.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-04T05:38:54.673 INFO:tasks.ceph.osd.5:Started 2023-12-04T05:38:54.673 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-04T05:38:54.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-04T05:38:54.675 INFO:tasks.ceph.osd.6:Started 2023-12-04T05:38:54.675 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-04T05:38:54.675 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-04T05:38:54.678 INFO:tasks.ceph.osd.7:Started 2023-12-04T05:38:54.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:38:54.705 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.704+0000 7f14d0213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-04T05:38:54.757 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.756+0000 7f868bc39040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-04T05:38:54.758 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.757+0000 7f14d0213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-04T05:38:54.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.793+0000 7f8c691a0640 1 Processor -- start 2023-12-04T05:38:54.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.793+0000 7f8c691a0640 1 -- start start 2023-12-04T05:38:54.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.794+0000 7f8c691a0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c6406bbe0 0x7f8c6406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.794+0000 7f8c691a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c64159750 0x7f8c640ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c691a0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c640cdaa0 con 0x7f8c64159370 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c691a0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c640cdd80 con 0x7f8c6406bbe0 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c691a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640ce060 con 0x7f8c64159750 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c63fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8c64159370 0x7f8c6406b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:50984/0 (socket says 172.21.15.111:50984) 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c63fff640 1 -- 172.21.15.111:0/3272343943 learned_addr learned my addr 172.21.15.111:0/3272343943 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c637fe640 1 --2- 172.21.15.111:0/3272343943 >> v2:172.21.15.111:3300/0 conn(0x7f8c6406bbe0 0x7f8c6406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c6899f640 1 --2- 172.21.15.111:0/3272343943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c64159750 0x7f8c640ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c637fe640 1 -- 172.21.15.111:0/3272343943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c64159750 msgr2=0x7f8c640ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c637fe640 1 --2- 172.21.15.111:0/3272343943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c64159750 0x7f8c640ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c637fe640 1 -- 172.21.15.111:0/3272343943 >> v1:172.21.15.111:6789/0 conn(0x7f8c64159370 legacy=0x7f8c6406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c637fe640 1 -- 172.21.15.111:0/3272343943 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640ce340 con 0x7f8c6406bbe0 2023-12-04T05:38:54.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.795+0000 7f8c62ffd640 1 -- 172.21.15.111:0/3272343943 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575654110 0 0) 0x7f8c640cdaa0 con 0x7f8c64159370 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c637fe640 1 --2- 172.21.15.111:0/3272343943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6406bbe0 0x7f8c6406d9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c580097a0 tx=0x7f8c5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223d39f6ba5c8b2c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c62ffd640 1 -- 172.21.15.111:0/3272343943 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58031830 con 0x7f8c6406bbe0 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c62ffd640 1 -- 172.21.15.111:0/3272343943 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58031990 con 0x7f8c6406bbe0 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c62ffd640 1 -- 172.21.15.111:0/3272343943 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58031d20 con 0x7f8c6406bbe0 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c691a0640 1 -- 172.21.15.111:0/3272343943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6406bbe0 msgr2=0x7f8c6406d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c691a0640 1 --2- 172.21.15.111:0/3272343943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6406bbe0 0x7f8c6406d9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c580097a0 tx=0x7f8c5802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c691a0640 1 -- 172.21.15.111:0/3272343943 shutdown_connections 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c691a0640 1 --2- 172.21.15.111:0/3272343943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c64159750 0x7f8c640ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c691a0640 1 --2- 172.21.15.111:0/3272343943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6406bbe0 0x7f8c6406d9a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.796+0000 7f8c691a0640 1 -- 172.21.15.111:0/3272343943 >> 172.21.15.111:0/3272343943 conn(0x7f8c64070a50 msgr2=0x7f8c64070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.797+0000 7f8c691a0640 1 -- 172.21.15.111:0/3272343943 shutdown_connections 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.797+0000 7f8c691a0640 1 -- 172.21.15.111:0/3272343943 wait complete. 2023-12-04T05:38:54.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.798+0000 7f8c691a0640 1 Processor -- start 2023-12-04T05:38:54.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.798+0000 7f8c691a0640 1 -- start start 2023-12-04T05:38:54.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.798+0000 7f8c691a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6406bbe0 0x7f8c641502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.798+0000 7f8c691a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 0x7f8c64147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.798+0000 7f8c691a0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c64159750 0x7f8c6414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c691a0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6414e940 con 0x7f8c64159370 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c691a0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6414e5f0 con 0x7f8c64159750 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c691a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6415c210 con 0x7f8c6406bbe0 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 0x7f8c64147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 0x7f8c64147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41300/0 (socket says 172.21.15.111:41300) 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 -- 172.21.15.111:0/822585851 learned_addr learned my addr 172.21.15.111:0/822585851 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c63fff640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6406bbe0 0x7f8c641502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c6899f640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c64159750 0x7f8c6414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 -- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6406bbe0 msgr2=0x7f8c641502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6406bbe0 0x7f8c641502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 -- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c64159750 msgr2=0x7f8c6414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c64159750 0x7f8c6414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 -- 172.21.15.111:0/822585851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c58009400 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c637fe640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 0x7f8c64147e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c5802bee0 tx=0x7f8c580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5803a6f0 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5803a870 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6415c4f0 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5803ac20 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6415be20 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8c5804d740 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.799+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8c58043c20 con 0x7f8c64159370 2023-12-04T05:38:54.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.800+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6414ebb0 con 0x7f8c64159370 2023-12-04T05:38:54.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.802+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8c58048070 con 0x7f8c64159370 2023-12-04T05:38:54.858 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.856+0000 7f868bc39040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-04T05:38:54.860 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.859+0000 7f14d0213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-04T05:38:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.863+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c64072490 con 0x7f8c64159370 2023-12-04T05:38:54.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c60ff9640 1 -- 172.21.15.111:0/822585851 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8c58035020 con 0x7f8c64159370 2023-12-04T05:38:54.865 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:38:54.865 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:38:54.612762+0000","last_up_change":"0.000000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 msgr2=0x7f8c64147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 0x7f8c64147e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c5802bee0 tx=0x7f8c580046c0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 shutdown_connections 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c64159750 0x7f8c6414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c64159370 0x7f8c64147e70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 --2- 172.21.15.111:0/822585851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6406bbe0 0x7f8c641502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.864+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 >> 172.21.15.111:0/822585851 conn(0x7f8c64070a50 msgr2=0x7f8c64070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.865+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 shutdown_connections 2023-12-04T05:38:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:38:54.865+0000 7f8c691a0640 1 -- 172.21.15.111:0/822585851 wait complete. 2023-12-04T05:38:54.876 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-04T05:38:54.877 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-04T05:38:54.954 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:54.953+0000 7f14d0213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-04T05:38:55.000 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:54.999+0000 7f868bc39040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-04T05:38:55.042 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:55.041+0000 7f868bc39040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-04T05:38:55.096 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:55.094+0000 7f14d0213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-04T05:38:55.138 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:55.137+0000 7f14d0213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-04T05:38:55.183 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:55.182+0000 7f868bc39040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-04T05:38:55.236 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:38:55.234+0000 7f15bb214500 -1 Falling back to public interface 2023-12-04T05:38:55.240 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:38:55.239+0000 7f8524016500 -1 Falling back to public interface 2023-12-04T05:38:55.260 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:38:55.258+0000 7fb41700f500 -1 Falling back to public interface 2023-12-04T05:38:55.260 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:38:55.258+0000 7f7d4d027500 -1 Falling back to public interface 2023-12-04T05:38:55.260 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:38:55.259+0000 7f7d4ba2c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-12-04T05:38:55.262 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:38:55.261+0000 7fb4159e6640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6828/0: (98) Address already in use 2023-12-04T05:38:55.273 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:38:55.272+0000 7f2b59216500 -1 Falling back to public interface 2023-12-04T05:38:55.274 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:38:55.273+0000 7fd88a28c500 -1 Falling back to public interface 2023-12-04T05:38:55.274 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:38:55.273+0000 7fd889233640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2023-12-04T05:38:55.280 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:55.279+0000 7f14d0213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-04T05:38:55.292 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:38:55.291+0000 7f71c3a12500 -1 Falling back to public interface 2023-12-04T05:38:55.293 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:38:55.292+0000 7fd9e7038500 -1 Falling back to public interface 2023-12-04T05:38:55.294 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:38:55.293+0000 7f71c23c2640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6821/0: (98) Address already in use 2023-12-04T05:38:55.682 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-04T05:38:55.780 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.778+0000 7fef8a883640 1 Processor -- start 2023-12-04T05:38:55.780 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.778+0000 7fef8a883640 1 -- start start 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef8a883640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fef8406ac20 0x7fef8415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef8a883640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef8a883640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef84071d20 con 0x7fef84144030 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef8a883640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef840ce340 con 0x7fef8406ac20 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef8a883640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef840ce7d0 con 0x7fef8406df20 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fef8406ac20 0x7fef8415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fef8406ac20 0x7fef8415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41316/0 (socket says 172.21.15.111:41316) 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 -- 172.21.15.111:0/2570883851 learned_addr learned my addr 172.21.15.111:0/2570883851 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 -- 172.21.15.111:0/2570883851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 msgr2=0x7fef8415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:55.781 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 --2- 172.21.15.111:0/2570883851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.782 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 -- 172.21.15.111:0/2570883851 >> v1:172.21.15.192:6789/0 conn(0x7fef84144030 legacy=0x7fef8406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:38:55.782 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.779+0000 7fef837fe640 1 -- 172.21.15.111:0/2570883851 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef84072250 con 0x7fef8406ac20 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.780+0000 7fef837fe640 1 --2- 172.21.15.111:0/2570883851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef8415ef20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef74009870 tx=0x7fef7402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08c5b10db2ec833 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.780+0000 7fef82ffd640 1 -- 172.21.15.111:0/2570883851 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74031740 con 0x7fef8406ac20 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.780+0000 7fef82ffd640 1 -- 172.21.15.111:0/2570883851 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef740318a0 con 0x7fef8406ac20 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.780+0000 7fef82ffd640 1 -- 172.21.15.111:0/2570883851 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74031c30 con 0x7fef8406ac20 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 -- 172.21.15.111:0/2570883851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 msgr2=0x7fef8415ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 --2- 172.21.15.111:0/2570883851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef8415ef20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef74009870 tx=0x7fef7402b7e0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 -- 172.21.15.111:0/2570883851 shutdown_connections 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 --2- 172.21.15.111:0/2570883851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.783 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 --2- 172.21.15.111:0/2570883851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef8415ef20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.784 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 -- 172.21.15.111:0/2570883851 >> 172.21.15.111:0/2570883851 conn(0x7fef8406c140 msgr2=0x7fef840b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:55.784 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 -- 172.21.15.111:0/2570883851 shutdown_connections 2023-12-04T05:38:55.784 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.781+0000 7fef8a883640 1 -- 172.21.15.111:0/2570883851 wait complete. 2023-12-04T05:38:55.786 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.782+0000 7fef8a883640 1 Processor -- start 2023-12-04T05:38:55.786 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.782+0000 7fef8a883640 1 -- start start 2023-12-04T05:38:55.786 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef8a883640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef840ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:55.786 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef8a883640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8413f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:55.786 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef8a883640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef84144030 0x7fef840a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:38:55.786 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef8a883640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef841f3390 con 0x7fef8406ac20 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef8a883640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef840a96c0 con 0x7fef84144030 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef8a883640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef840a99a0 con 0x7fef8406df20 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef83fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef840ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef83fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef840ab3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41318/0 (socket says 172.21.15.111:41318) 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef83fff640 1 -- 172.21.15.111:0/631460769 learned_addr learned my addr 172.21.15.111:0/631460769 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef837fe640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8413f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.783+0000 7fef88e20640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef84144030 0x7fef840a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef83fff640 1 -- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 msgr2=0x7fef8413f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef83fff640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8413f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef83fff640 1 -- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef84144030 msgr2=0x7fef840a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef83fff640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef84144030 0x7fef840a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef83fff640 1 -- 172.21.15.111:0/631460769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740094d0 con 0x7fef8406ac20 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef83fff640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef840ab3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef7800c430 tx=0x7fef7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:38:55.787 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78011790 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef780118f0 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef840a9c80 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78011ce0 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8413fc10 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fef7801c760 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.784+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fef7801b3c0 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.785+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8415b910 con 0x7fef8406ac20 2023-12-04T05:38:55.788 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.787+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fef78011e40 con 0x7fef8406ac20 2023-12-04T05:38:55.850 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.848+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef84067460 con 0x7fef8406ac20 2023-12-04T05:38:55.850 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.849+0000 7fef80ff9640 1 -- 172.21.15.111:0/631460769 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fef7802b090 con 0x7fef8406ac20 2023-12-04T05:38:55.850 INFO:teuthology.misc.health.smithi111.stdout: 2023-12-04T05:38:55.850 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":10,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:38:54.612762+0000","last_up_change":"0.000000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:38:55.851 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 msgr2=0x7fef840ab3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef840ab3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef7800c430 tx=0x7fef7800c8f0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 shutdown_connections 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef84144030 0x7fef840a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef8406df20 0x7fef8413f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 --2- 172.21.15.111:0/631460769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef8406ac20 0x7fef840ab3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 >> 172.21.15.111:0/631460769 conn(0x7fef8406c140 msgr2=0x7fef84140640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 shutdown_connections 2023-12-04T05:38:55.852 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:38:55.850+0000 7fef8a883640 1 -- 172.21.15.111:0/631460769 wait complete. 2023-12-04T05:38:55.860 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-04T05:38:56.027 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:56.026+0000 7f868bc39040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-04T05:38:56.095 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:56.094+0000 7f14d0213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-04T05:38:56.297 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:56.296+0000 7f868bc39040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-04T05:38:56.349 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:56.348+0000 7f868bc39040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-04T05:38:56.365 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:56.364+0000 7f14d0213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-04T05:38:56.417 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:56.416+0000 7f14d0213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-04T05:38:56.452 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:56.450+0000 7f868bc39040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-04T05:38:56.523 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-04T05:38:56.522+0000 7f14d0213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-04T05:38:57.478 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:57.475+0000 7f862d011640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-04T05:38:57.478 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:38:57.476+0000 7f862d011640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-04T05:38:58.713 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:38:58.712+0000 7f8524016500 -1 osd.1 0 log_to_monitors true 2023-12-04T05:38:58.743 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:38:58.741+0000 7f15bb214500 -1 osd.0 0 log_to_monitors true 2023-12-04T05:38:58.791 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:38:58.789+0000 7fb41700f500 -1 osd.3 0 log_to_monitors true 2023-12-04T05:38:58.795 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:38:58.794+0000 7f7d4d027500 -1 osd.2 0 log_to_monitors true 2023-12-04T05:38:58.806 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:38:58.805+0000 7fd9e7038500 -1 osd.7 0 log_to_monitors true 2023-12-04T05:38:58.807 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:38:58.806+0000 7fd88a28c500 -1 osd.4 0 log_to_monitors true 2023-12-04T05:38:58.811 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:38:58.810+0000 7f71c3a12500 -1 osd.6 0 log_to_monitors true 2023-12-04T05:38:58.812 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:38:58.811+0000 7f2b59216500 -1 osd.5 0 log_to_monitors true 2023-12-04T05:39:02.541 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:39:02.540+0000 7f71beb86640 -1 osd.6 0 waiting for initial osdmap 2023-12-04T05:39:02.549 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:39:02.548+0000 7fd9e3fdd640 -1 osd.7 0 waiting for initial osdmap 2023-12-04T05:39:02.557 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:39:02.556+0000 7f71b71bf640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.567 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:39:02.566+0000 7fd9dadde640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.569 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:39:02.568+0000 7f2b542c3640 -1 osd.5 0 waiting for initial osdmap 2023-12-04T05:39:02.581 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:39:02.580+0000 7fd8859f7640 -1 osd.4 0 waiting for initial osdmap 2023-12-04T05:39:02.584 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:39:02.583+0000 7f2b4c8fc640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.593 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:39:02.592+0000 7fd87e030640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.635 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:39:02.633+0000 7f7d481f0640 -1 osd.2 0 waiting for initial osdmap 2023-12-04T05:39:02.651 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:39:02.650+0000 7f7d40829640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.666 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:02.735 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:39:02.734+0000 7f851f1e2640 -1 osd.1 0 waiting for initial osdmap 2023-12-04T05:39:02.749 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:39:02.748+0000 7f851781b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.770 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:39:02.769+0000 7f15b63fe640 -1 osd.0 0 waiting for initial osdmap 2023-12-04T05:39:02.783 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:39:02.782+0000 7f15aea37640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.794 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.793+0000 7fd73e690640 1 Processor -- start 2023-12-04T05:39:02.795 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.793+0000 7fd73e690640 1 -- start start 2023-12-04T05:39:02.796 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.794+0000 7fd73e690640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7380b43e0 0x7fd7380d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:02.796 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.794+0000 7fd73e690640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7380d1940 con 0x7fd7380d2e90 2023-12-04T05:39:02.796 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.794+0000 7fd73e690640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73815d8b0 con 0x7fd73815d4d0 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.794+0000 7fd73e690640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380d0b80 con 0x7fd7380b43e0 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.794+0000 7fd73cc2d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd7380d2e90 0x7fd7380d3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51520/0 (socket says 172.21.15.111:51520) 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.794+0000 7fd73cc2d640 1 -- 172.21.15.111:0/3377966856 learned_addr learned my addr 172.21.15.111:0/3377966856 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd7377fe640 1 --2- 172.21.15.111:0/3377966856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7380b43e0 0x7fd7380d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300577318 0 0) 0x7fd730001490 con 0x7fd7380d2e90 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd720003520 con 0x7fd7380d2e90 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162808299 0 0) 0x7fd73815d8b0 con 0x7fd73815d4d0 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd730001490 con 0x7fd73815d4d0 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd7377fe640 1 -- 172.21.15.111:0/3377966856 >> v1:172.21.15.192:6789/0 conn(0x7fd73815d4d0 legacy=0x7fd7380cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd7377fe640 1 -- 172.21.15.111:0/3377966856 >> v1:172.21.15.111:6789/0 conn(0x7fd7380d2e90 legacy=0x7fd7380d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.797 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.795+0000 7fd7377fe640 1 -- 172.21.15.111:0/3377966856 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380b2ee0 con 0x7fd7380b43e0 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.796+0000 7fd7377fe640 1 --2- 172.21.15.111:0/3377966856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7380b43e0 0x7fd7380d0640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0097d0 tx=0x7fd72c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=653166dc87ec4148 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.796+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c031800 con 0x7fd7380b43e0 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.796+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c031960 con 0x7fd7380b43e0 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.796+0000 7fd736ffd640 1 -- 172.21.15.111:0/3377966856 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c031cf0 con 0x7fd7380b43e0 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 -- 172.21.15.111:0/3377966856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7380b43e0 msgr2=0x7fd7380d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 --2- 172.21.15.111:0/3377966856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7380b43e0 0x7fd7380d0640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0097d0 tx=0x7fd72c02b8a0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 -- 172.21.15.111:0/3377966856 shutdown_connections 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 --2- 172.21.15.111:0/3377966856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7380b43e0 0x7fd7380d0640 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 -- 172.21.15.111:0/3377966856 >> 172.21.15.111:0/3377966856 conn(0x7fd73808af90 msgr2=0x7fd73809c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 -- 172.21.15.111:0/3377966856 shutdown_connections 2023-12-04T05:39:02.799 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.797+0000 7fd73e690640 1 -- 172.21.15.111:0/3377966856 wait complete. 2023-12-04T05:39:02.801 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.798+0000 7fd73e690640 1 Processor -- start 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.798+0000 7fd73e690640 1 -- start start 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73e690640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7380b43e0 0x7fd7380b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73e690640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 0x7fd7380cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73e690640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd73815d4d0 0x7fd73815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73e690640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380b8b00 con 0x7fd7380d2e90 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73e690640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738153b60 con 0x7fd7380b43e0 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73e690640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380cb510 con 0x7fd73815d4d0 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 0x7fd7380cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 0x7fd7380cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43398/0 (socket says 172.21.15.111:43398) 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 -- 172.21.15.111:0/125394750 learned_addr learned my addr 172.21.15.111:0/125394750 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd737fff640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7380b43e0 0x7fd7380b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:02.802 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd73cc2d640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd73815d4d0 0x7fd73815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 -- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd73815d4d0 msgr2=0x7fd73815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd73815d4d0 0x7fd73815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 -- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7380b43e0 msgr2=0x7fd7380b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7380b43e0 0x7fd7380b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 -- 172.21.15.111:0/125394750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c009400 con 0x7fd7380d2e90 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.799+0000 7fd7377fe640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 0x7fd7380cc5b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0098f0 tx=0x7fd72c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.800+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c03bc70 con 0x7fd7380d2e90 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.800+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c03bdd0 con 0x7fd7380d2e90 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.800+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c045630 con 0x7fd7380d2e90 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.800+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7380cb7f0 con 0x7fd7380d2e90 2023-12-04T05:39:02.803 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.800+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7380cbdb0 con 0x7fd7380d2e90 2023-12-04T05:39:02.804 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.801+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd738069440 con 0x7fd7380d2e90 2023-12-04T05:39:02.804 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.801+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 89493+0+0 (secure 0 0 0) 0x7fd72c04e450 con 0x7fd7380d2e90 2023-12-04T05:39:02.804 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.802+0000 7fd734ff9640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd72406b240 0x7fd72406d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:02.804 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.802+0000 7fd737fff640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd72406b240 0x7fd72406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:02.804 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.802+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd72c02e070 con 0x7fd7380d2e90 2023-12-04T05:39:02.804 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.802+0000 7fd737fff640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd72406b240 0x7fd72406d700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd728000f60 tx=0x7fd728009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:02.808 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.806+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd72c076cf0 con 0x7fd7380d2e90 2023-12-04T05:39:02.845 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:39:02.843+0000 7fb4121aa640 -1 osd.3 0 waiting for initial osdmap 2023-12-04T05:39:02.861 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:39:02.860+0000 7fb40a7e3640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-04T05:39:02.991 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.990+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd738069440 con 0x7fd7380d2e90 2023-12-04T05:39:02.992 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.990+0000 7fd734ff9640 1 -- 172.21.15.111:0/125394750 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd72c035020 con 0x7fd7380d2e90 2023-12-04T05:39:02.992 INFO:teuthology.misc.health.smithi111.stdout: 2023-12-04T05:39:02.992 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":12,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:00.477540+0000","last_up_change":"0.000000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:02.994 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd72406b240 msgr2=0x7fd72406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.994 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd72406b240 0x7fd72406d700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd728000f60 tx=0x7fd728009040 comp rx=0 tx=0).stop 2023-12-04T05:39:02.994 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 msgr2=0x7fd7380cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:02.994 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 0x7fd7380cc5b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0098f0 tx=0x7fd72c0046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.994 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 shutdown_connections 2023-12-04T05:39:02.994 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd72406b240 0x7fd72406d700 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.995 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd73815d4d0 0x7fd73815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.995 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7380d2e90 0x7fd7380cc5b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.995 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 --2- 172.21.15.111:0/125394750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7380b43e0 0x7fd7380b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:02.995 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.992+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 >> 172.21.15.111:0/125394750 conn(0x7fd73808af90 msgr2=0x7fd7380cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:02.995 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.993+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 shutdown_connections 2023-12-04T05:39:02.995 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:02.993+0000 7fd73e690640 1 -- 172.21.15.111:0/125394750 wait complete. 2023-12-04T05:39:03.003 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-04T05:39:09.811 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:09.910 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.909+0000 7fbc40e51640 1 Processor -- start 2023-12-04T05:39:09.911 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.909+0000 7fbc40e51640 1 -- start start 2023-12-04T05:39:09.913 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc40e51640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbc3c1415f0 0x7fbc3c06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.913 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc40e51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 0x7fbc3c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc40e51640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c15c7d0 con 0x7fbc3c06ba60 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc40e51640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c1419d0 con 0x7fbc3c1415f0 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc40e51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b3600 con 0x7fbc3c0cdad0 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc39d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbc3c06ba60 0x7fbc3c0cd3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51534/0 (socket says 172.21.15.111:51534) 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc39d9b640 1 -- 172.21.15.111:0/2717047257 learned_addr learned my addr 172.21.15.111:0/2717047257 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777598051 0 0) 0x7fbc3c15c7d0 con 0x7fbc3c06ba60 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/2717047257 >> v2:172.21.15.192:3300/0 conn(0x7fbc3c1415f0 0x7fbc3c06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc24003520 con 0x7fbc3c06ba60 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.910+0000 7fbc3ad9d640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 0x7fbc3c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495070174 0 0) 0x7fbc24003520 con 0x7fbc3c06ba60 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 msgr2=0x7fbc3c0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 0x7fbc3c0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.914 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 >> v2:172.21.15.192:3300/0 conn(0x7fbc3c1415f0 msgr2=0x7fbc3c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 --2- 172.21.15.111:0/2717047257 >> v2:172.21.15.192:3300/0 conn(0x7fbc3c1415f0 0x7fbc3c06b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c06bec0 con 0x7fbc3c06ba60 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fbc30001670 con 0x7fbc3c06ba60 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 >> v1:172.21.15.111:6789/0 conn(0x7fbc3c06ba60 legacy=0x7fbc3c0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc24003970 0x7fbc24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc240047c0 0x7fbc24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 0x7fbc24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c1419d0 con 0x7fbc24003970 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b3600 con 0x7fbc240047c0 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2400b9b0 con 0x7fbc3c0cdad0 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc24003970 0x7fbc24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc39d9b640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc240047c0 0x7fbc24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3ad9d640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 0x7fbc24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3a59c640 1 -- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 msgr2=0x7fbc24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:09.915 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc3c0cdad0 0x7fbc24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3a59c640 1 -- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc240047c0 msgr2=0x7fbc24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc240047c0 0x7fbc24004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.911+0000 7fbc3a59c640 1 -- 172.21.15.111:0/2717047257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24003520 con 0x7fbc24003970 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.912+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc24003970 0x7fbc24003d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009fc0 tx=0x7fbc240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcce079981759452 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.912+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c02cc40 con 0x7fbc24003970 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.912+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c00f040 con 0x7fbc24003970 2023-12-04T05:39:09.916 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.912+0000 7fbc3959a640 1 -- 172.21.15.111:0/2717047257 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c034440 con 0x7fbc24003970 2023-12-04T05:39:09.919 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc40e51640 1 -- 172.21.15.111:0/2717047257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc24003970 msgr2=0x7fbc24003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:09.919 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc40e51640 1 --2- 172.21.15.111:0/2717047257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc24003970 0x7fbc24003d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009fc0 tx=0x7fbc240057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.919 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc3ad9d640 1 -- 172.21.15.111:0/2717047257 reap_dead start 2023-12-04T05:39:09.919 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc40e51640 1 -- 172.21.15.111:0/2717047257 shutdown_connections 2023-12-04T05:39:09.919 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc40e51640 1 -- 172.21.15.111:0/2717047257 >> 172.21.15.111:0/2717047257 conn(0x7fbc3c070f40 msgr2=0x7fbc3c071350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc40e51640 1 -- 172.21.15.111:0/2717047257 shutdown_connections 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.913+0000 7fbc40e51640 1 -- 172.21.15.111:0/2717047257 wait complete. 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.914+0000 7fbc40e51640 1 Processor -- start 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.914+0000 7fbc40e51640 1 -- start start 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc40e51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc24003970 0x7fbc3c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc40e51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 0x7fbc3c13c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc40e51640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc3c137500 0x7fbc3c137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc40e51640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c138110 con 0x7fbc3c13c460 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc40e51640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c13ce20 con 0x7fbc3c137500 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc40e51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c13a1e0 con 0x7fbc24003970 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 0x7fbc3c13c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 0x7fbc3c13c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43434/0 (socket says 172.21.15.111:43434) 2023-12-04T05:39:09.920 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 -- 172.21.15.111:0/3673348992 learned_addr learned my addr 172.21.15.111:0/3673348992 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc24003970 0x7fbc3c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc3ad9d640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc3c137500 0x7fbc3c137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 -- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc24003970 msgr2=0x7fbc3c155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc24003970 0x7fbc3c155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 -- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc3c137500 msgr2=0x7fbc3c137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc3c137500 0x7fbc3c137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 -- 172.21.15.111:0/3673348992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c13b2f0 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.915+0000 7fbc39d9b640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 0x7fbc3c13c8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc30007340 tx=0x7fbc30007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.916+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000f8c0 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.916+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3000fa40 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.916+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000fdf0 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.916+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c139d50 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.916+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c137ec0 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.917+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c139ee0 con 0x7fbc3c13c460 2023-12-04T05:39:09.921 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.917+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbc3001d460 con 0x7fbc3c13c460 2023-12-04T05:39:09.922 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.917+0000 7fbc2affd640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc1c06b2a0 0x7fbc1c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:09.922 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.918+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc30089aa0 con 0x7fbc3c13c460 2023-12-04T05:39:09.922 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.918+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc1c06b2a0 0x7fbc1c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:09.922 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.918+0000 7fbc3a59c640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc1c06b2a0 0x7fbc1c06d760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c036000 tx=0x7fbc2c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:09.923 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:09.922+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc30054c60 con 0x7fbc3c13c460 2023-12-04T05:39:10.090 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.089+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc3c069440 con 0x7fbc3c13c460 2023-12-04T05:39:10.091 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.090+0000 7fbc2affd640 1 -- 172.21.15.111:0/3673348992 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12826 (secure 0 0 0) 0x7fbc30058280 con 0x7fbc3c13c460 2023-12-04T05:39:10.091 INFO:teuthology.misc.health.smithi111.stdout: 2023-12-04T05:39:10.091 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":16,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:06.563565+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc1c06b2a0 msgr2=0x7fbc1c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc1c06b2a0 0x7fbc1c06d760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c036000 tx=0x7fbc2c02e000 comp rx=0 tx=0).stop 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 msgr2=0x7fbc3c13c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 0x7fbc3c13c8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc30007340 tx=0x7fbc30007800 comp rx=0 tx=0).stop 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 shutdown_connections 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc1c06b2a0 0x7fbc1c06d760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc3c137500 0x7fbc3c137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc3c13c460 0x7fbc3c13c8e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.093 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 --2- 172.21.15.111:0/3673348992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc24003970 0x7fbc3c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.094 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 >> 172.21.15.111:0/3673348992 conn(0x7fbc3c070f40 msgr2=0x7fbc3c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:10.094 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 shutdown_connections 2023-12-04T05:39:10.094 INFO:teuthology.misc.health.smithi111.stderr:2023-12-04T05:39:10.091+0000 7fbc40e51640 1 -- 172.21.15.111:0/3673348992 wait complete. 2023-12-04T05:39:10.102 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-04T05:39:10.102 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-04T05:39:10.103 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-04T05:39:10.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-04T05:39:10.105 INFO:tasks.ceph.mds.a:Started 2023-12-04T05:39:10.105 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-04T05:39:10.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-04T05:39:10.107 INFO:tasks.ceph.mds.c:Started 2023-12-04T05:39:10.107 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-04T05:39:10.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-04T05:39:10.110 INFO:tasks.ceph.mds.b:Started 2023-12-04T05:39:10.110 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-04T05:39:10.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-04T05:39:10.112 INFO:tasks.ceph.mds.d:Started 2023-12-04T05:39:10.113 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-04T05:39:10.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:10.180 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2023-12-04T05:39:10.180 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2023-12-04T05:39:10.194 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-04T05:39:10.195 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-04T05:39:10.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.217+0000 7f21fb5be640 1 Processor -- start 2023-12-04T05:39:10.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.217+0000 7f21fb5be640 1 -- start start 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21fb5be640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21f40d1780 0x7f21f40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21fb5be640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f21f40b20a0 0x7f21f40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21fb5be640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f40cdcb0 con 0x7f21f40b2ad0 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21fb5be640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f40d20a0 con 0x7f21f40d1780 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21fb5be640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f40d39e0 con 0x7f21f40b20a0 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21f40d1780 0x7f21f40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f21f40d1780 0x7f21f40d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43448/0 (socket says 172.21.15.111:43448) 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 -- 172.21.15.111:0/4077643047 learned_addr learned my addr 172.21.15.111:0/4077643047 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 -- 172.21.15.111:0/4077643047 >> v2:172.21.15.192:3300/0 conn(0x7f21f40b20a0 msgr2=0x7f21f40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 --2- 172.21.15.111:0/4077643047 >> v2:172.21.15.192:3300/0 conn(0x7f21f40b20a0 0x7f21f40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 -- 172.21.15.111:0/4077643047 >> v1:172.21.15.192:6789/0 conn(0x7f21f40b2ad0 legacy=0x7f21f40ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:10.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.218+0000 7f21f935a640 1 -- 172.21.15.111:0/4077643047 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40d3cc0 con 0x7f21f40d1780 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.219+0000 7f21f935a640 1 --2- 172.21.15.111:0/4077643047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40d01e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21f40d0c50 tx=0x7f21f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9549db73f60c3d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.219+0000 7f21e7fff640 1 -- 172.21.15.111:0/4077643047 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00317e0 con 0x7f21f40d1780 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.219+0000 7f21e7fff640 1 -- 172.21.15.111:0/4077643047 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0031940 con 0x7f21f40d1780 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.219+0000 7f21e7fff640 1 -- 172.21.15.111:0/4077643047 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0031cd0 con 0x7f21f40d1780 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 -- 172.21.15.111:0/4077643047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 msgr2=0x7f21f40d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 --2- 172.21.15.111:0/4077643047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40d01e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21f40d0c50 tx=0x7f21f002b880 comp rx=0 tx=0).stop 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 -- 172.21.15.111:0/4077643047 shutdown_connections 2023-12-04T05:39:10.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 --2- 172.21.15.111:0/4077643047 >> v2:172.21.15.192:3300/0 conn(0x7f21f40b20a0 0x7f21f40d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 --2- 172.21.15.111:0/4077643047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40d01e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 -- 172.21.15.111:0/4077643047 >> 172.21.15.111:0/4077643047 conn(0x7f21f409c4e0 msgr2=0x7f21f40d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:10.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 -- 172.21.15.111:0/4077643047 shutdown_connections 2023-12-04T05:39:10.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.220+0000 7f21fb5be640 1 -- 172.21.15.111:0/4077643047 wait complete. 2023-12-04T05:39:10.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.221+0000 7f21fb5be640 1 Processor -- start 2023-12-04T05:39:10.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.221+0000 7f21fb5be640 1 -- start start 2023-12-04T05:39:10.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.221+0000 7f21fb5be640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21f40b20a0 0x7f21f4154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.221+0000 7f21fb5be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21f40b2ad0 0x7f21f40ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21fb5be640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21fb5be640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f40ca410 con 0x7f21f40d1780 2023-12-04T05:39:10.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21fb5be640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f40ca6f0 con 0x7f21f40b20a0 2023-12-04T05:39:10.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21fb5be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f40ca9d0 con 0x7f21f40b2ad0 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43464/0 (socket says 172.21.15.111:43464) 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 -- 172.21.15.111:0/2700478200 learned_addr learned my addr 172.21.15.111:0/2700478200 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f935a640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21f40b20a0 0x7f21f4154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f8b59640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21f40b2ad0 0x7f21f40ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 -- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21f40b2ad0 msgr2=0x7f21f40ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21f40b2ad0 0x7f21f40ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 -- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21f40b20a0 msgr2=0x7f21f4154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21f40b20a0 0x7f21f4154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 -- 172.21.15.111:0/2700478200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f00094d0 con 0x7f21f40d1780 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21f9b5b640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40cb4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b990 tx=0x7f21ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00cba0 con 0x7f21f40d1780 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec00cd00 con 0x7f21f40d1780 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec014680 con 0x7f21f40d1780 2023-12-04T05:39:10.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f40cacb0 con 0x7f21f40d1780 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.222+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f40cb9f0 con 0x7f21f40d1780 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.223+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f40cb1d0 con 0x7f21f40d1780 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.224+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f21ec0147e0 con 0x7f21f40d1780 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.224+0000 7f21e5ffb640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f21d806b2a0 0x7f21d806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.224+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f21ec089710 con 0x7f21f40d1780 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.224+0000 7f21f935a640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f21d806b2a0 0x7f21d806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.225+0000 7f21f935a640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f21d806b2a0 0x7f21d806d760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21f0036000 tx=0x7f21f003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.229+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21f40cb1d0 con 0x7f21f40d1780 2023-12-04T05:39:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.395+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21f40cb1d0 con 0x7f21f40d1780 2023-12-04T05:39:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.395+0000 7f21e5ffb640 1 -- 172.21.15.111:0/2700478200 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f21f40cb1d0 con 0x7f21f40d1780 2023-12-04T05:39:10.399 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:10.399 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:06.563565+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f21d806b2a0 msgr2=0x7f21d806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f21d806b2a0 0x7f21d806d760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21f0036000 tx=0x7f21f003d040 comp rx=0 tx=0).stop 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 msgr2=0x7f21f40cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40cb4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b990 tx=0x7f21ec00be50 comp rx=0 tx=0).stop 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 shutdown_connections 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f21d806b2a0 0x7f21d806d760 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21f40d1780 0x7f21f40cb4b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21f40b2ad0 0x7f21f40ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 --2- 172.21.15.111:0/2700478200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21f40b20a0 0x7f21f4154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 >> 172.21.15.111:0/2700478200 conn(0x7f21f409c4e0 msgr2=0x7f21f415b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:10.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 shutdown_connections 2023-12-04T05:39:10.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.397+0000 7f21fb5be640 1 -- 172.21.15.111:0/2700478200 wait complete. 2023-12-04T05:39:10.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:10.410 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:10.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.511+0000 7f97e4656640 1 Processor -- start 2023-12-04T05:39:10.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.511+0000 7f97e4656640 1 -- start start 2023-12-04T05:39:10.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e4656640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97dc140300 0x7f97dc0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e4656640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97dc0b34a0 0x7f97dc06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e4656640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97dc15d7f0 con 0x7f97dc0b3030 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e4656640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc0b4e20 con 0x7f97dc140300 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e4656640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc1406e0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97dc140300 0x7f97dc0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97dc140300 0x7f97dc0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47294/0 (socket says 172.21.15.111:47294) 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 -- 172.21.15.111:0/1382351045 learned_addr learned my addr 172.21.15.111:0/1382351045 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e2bf3640 1 --2- 172.21.15.111:0/1382351045 >> v2:172.21.15.192:3300/0 conn(0x7f97dc0b34a0 0x7f97dc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 -- 172.21.15.111:0/1382351045 >> v2:172.21.15.192:3300/0 conn(0x7f97dc0b34a0 msgr2=0x7f97dc06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 --2- 172.21.15.111:0/1382351045 >> v2:172.21.15.192:3300/0 conn(0x7f97dc0b34a0 0x7f97dc06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 -- 172.21.15.111:0/1382351045 >> v1:172.21.15.192:6789/0 conn(0x7f97dc0b3030 legacy=0x7f97dc06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.512+0000 7f97e23f2640 1 -- 172.21.15.111:0/1382351045 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc15dac0 con 0x7f97dc140300 2023-12-04T05:39:10.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e23f2640 1 --2- 172.21.15.111:0/1382351045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc140300 0x7f97dc0b4490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97dc070960 tx=0x7f97d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936ad40e47671a73 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e13f0640 1 -- 172.21.15.111:0/1382351045 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8031870 con 0x7f97dc140300 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e13f0640 1 -- 172.21.15.111:0/1382351045 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d80319d0 con 0x7f97dc140300 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e13f0640 1 -- 172.21.15.111:0/1382351045 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8031d60 con 0x7f97dc140300 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e4656640 1 -- 172.21.15.111:0/1382351045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc140300 msgr2=0x7f97dc0b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e4656640 1 --2- 172.21.15.111:0/1382351045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc140300 0x7f97dc0b4490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97dc070960 tx=0x7f97d802b910 comp rx=0 tx=0).stop 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e4656640 1 -- 172.21.15.111:0/1382351045 shutdown_connections 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e4656640 1 --2- 172.21.15.111:0/1382351045 >> v2:172.21.15.192:3300/0 conn(0x7f97dc0b34a0 0x7f97dc06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e4656640 1 --2- 172.21.15.111:0/1382351045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc140300 0x7f97dc0b4490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.513+0000 7f97e4656640 1 -- 172.21.15.111:0/1382351045 >> 172.21.15.111:0/1382351045 conn(0x7f97dc06a8b0 msgr2=0x7f97dc06acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.514+0000 7f97e4656640 1 -- 172.21.15.111:0/1382351045 shutdown_connections 2023-12-04T05:39:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.514+0000 7f97e4656640 1 -- 172.21.15.111:0/1382351045 wait complete. 2023-12-04T05:39:10.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 Processor -- start 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 -- start start 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97dc0b3030 0x7f97dc1f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 0x7f97dc1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97dc140300 0x7f97dc145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1f32e0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc156da0 con 0x7f97dc140300 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e4656640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc157080 con 0x7f97dc0b3030 2023-12-04T05:39:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e1bf1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 0x7f97dc1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e1bf1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 0x7f97dc1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47302/0 (socket says 172.21.15.111:47302) 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.515+0000 7f97e1bf1640 1 -- 172.21.15.111:0/2598957610 learned_addr learned my addr 172.21.15.111:0/2598957610 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e23f2640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97dc0b3030 0x7f97dc1f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e2bf3640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97dc140300 0x7f97dc145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e1bf1640 1 -- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97dc0b3030 msgr2=0x7f97dc1f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e1bf1640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97dc0b3030 0x7f97dc1f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e1bf1640 1 -- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97dc140300 msgr2=0x7f97dc145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e1bf1640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97dc140300 0x7f97dc145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e1bf1640 1 -- 172.21.15.111:0/2598957610 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80094d0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e1bf1640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 0x7f97dc1563c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c400 tx=0x7f97cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0052d0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc005430 con 0x7f97dc0b34a0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc003830 con 0x7f97dc0b34a0 2023-12-04T05:39:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc157360 con 0x7f97dc0b34a0 2023-12-04T05:39:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.516+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc157920 con 0x7f97dc0b34a0 2023-12-04T05:39:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.517+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc142e70 con 0x7f97dc0b34a0 2023-12-04T05:39:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.517+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f97cc003990 con 0x7f97dc0b34a0 2023-12-04T05:39:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.518+0000 7f97d2ffd640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f97c006b2a0 0x7f97c006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.518+0000 7f97e23f2640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f97c006b2a0 0x7f97c006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.518+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f97cc088ce0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.518+0000 7f97e23f2640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f97c006b2a0 0x7f97c006d760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97d8036000 tx=0x7f97d803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.522+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97cc053f50 con 0x7f97dc0b34a0 2023-12-04T05:39:10.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.691+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97dc13a7b0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.691+0000 7f97d2ffd640 1 -- 172.21.15.111:0/2598957610 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97cc0170a0 con 0x7f97dc0b34a0 2023-12-04T05:39:10.693 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f97c006b2a0 msgr2=0x7f97c006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f97c006b2a0 0x7f97c006d760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97d8036000 tx=0x7f97d803c040 comp rx=0 tx=0).stop 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 msgr2=0x7f97dc1563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 0x7f97dc1563c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c400 tx=0x7f97cc00c8c0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 shutdown_connections 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f97c006b2a0 0x7f97c006d760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97dc140300 0x7f97dc145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97dc0b34a0 0x7f97dc1563c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 --2- 172.21.15.111:0/2598957610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97dc0b3030 0x7f97dc1f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 >> 172.21.15.111:0/2598957610 conn(0x7f97dc06a8b0 msgr2=0x7f97dc15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 shutdown_connections 2023-12-04T05:39:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.693+0000 7f97e4656640 1 -- 172.21.15.111:0/2598957610 wait complete. 2023-12-04T05:39:10.704 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:10.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 Processor -- start 2023-12-04T05:39:10.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 -- start start 2023-12-04T05:39:10.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f999415b480 0x7f99940b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99940cf9f0 0x7f99940cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99940cbca0 con 0x7f999415edd0 2023-12-04T05:39:10.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99940cbf40 con 0x7f99940cf9f0 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.803+0000 7f9999bd5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99940cc220 con 0x7f999415b480 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99940cf9f0 0x7f99940cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99940cf9f0 0x7f99940cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47306/0 (socket says 172.21.15.111:47306) 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 -- 172.21.15.111:0/3091048864 learned_addr learned my addr 172.21.15.111:0/3091048864 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f99937fe640 1 --2- 172.21.15.111:0/3091048864 >> v2:172.21.15.192:3300/0 conn(0x7f999415b480 0x7f99940b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 -- 172.21.15.111:0/3091048864 >> v2:172.21.15.192:3300/0 conn(0x7f999415b480 msgr2=0x7f99940b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 --2- 172.21.15.111:0/3091048864 >> v2:172.21.15.192:3300/0 conn(0x7f999415b480 0x7f99940b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 -- 172.21.15.111:0/3091048864 >> v1:172.21.15.192:6789/0 conn(0x7f999415edd0 legacy=0x7f999415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 -- 172.21.15.111:0/3091048864 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940cc500 con 0x7f99940cf9f0 2023-12-04T05:39:10.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f99937fe640 1 --2- 172.21.15.111:0/3091048864 >> v2:172.21.15.192:3300/0 conn(0x7f999415b480 0x7f99940b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.804+0000 7f9992ffd640 1 --2- 172.21.15.111:0/3091048864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940cf9f0 0x7f99940cca20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9994071e00 tx=0x7f998802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2685cfbf9c39fa1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f99927fc640 1 -- 172.21.15.111:0/3091048864 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988031870 con 0x7f99940cf9f0 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f99927fc640 1 -- 172.21.15.111:0/3091048864 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880319d0 con 0x7f99940cf9f0 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f99927fc640 1 -- 172.21.15.111:0/3091048864 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988031d60 con 0x7f99940cf9f0 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 -- 172.21.15.111:0/3091048864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940cf9f0 msgr2=0x7f99940cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 --2- 172.21.15.111:0/3091048864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940cf9f0 0x7f99940cca20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9994071e00 tx=0x7f998802b910 comp rx=0 tx=0).stop 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 -- 172.21.15.111:0/3091048864 shutdown_connections 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 --2- 172.21.15.111:0/3091048864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940cf9f0 0x7f99940cca20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 --2- 172.21.15.111:0/3091048864 >> v2:172.21.15.192:3300/0 conn(0x7f999415b480 0x7f99940b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 -- 172.21.15.111:0/3091048864 >> 172.21.15.111:0/3091048864 conn(0x7f99940b2a70 msgr2=0x7f99940b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:10.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 -- 172.21.15.111:0/3091048864 shutdown_connections 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.805+0000 7f9999bd5640 1 -- 172.21.15.111:0/3091048864 wait complete. 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.806+0000 7f9999bd5640 1 Processor -- start 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 -- start start 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999415b480 0x7f9994159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f999415edd0 0x7f999415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 0x7f9994159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b6350 con 0x7f99940b5b40 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999415d6a0 con 0x7f999415edd0 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9999bd5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999415c3a0 con 0x7f999415b480 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9993fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 0x7f9994159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9993fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 0x7f9994159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47312/0 (socket says 172.21.15.111:47312) 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9993fff640 1 -- 172.21.15.111:0/627281691 learned_addr learned my addr 172.21.15.111:0/627281691 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.807+0000 7f9992ffd640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f999415edd0 0x7f999415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f99937fe640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999415b480 0x7f9994159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9993fff640 1 -- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999415b480 msgr2=0x7f9994159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9993fff640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999415b480 0x7f9994159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9993fff640 1 -- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f999415edd0 msgr2=0x7f999415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9993fff640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f999415edd0 0x7f999415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9993fff640 1 -- 172.21.15.111:0/627281691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880094d0 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9992ffd640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f999415edd0 0x7f999415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9993fff640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 0x7f9994159610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b970 tx=0x7f998c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c00cb40 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c00cca0 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c014680 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999415c680 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.808+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999415cc40 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.809+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999415ce60 con 0x7f99940b5b40 2023-12-04T05:39:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.809+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f998c01d3b0 con 0x7f99940b5b40 2023-12-04T05:39:10.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.810+0000 7f997bfff640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f998006b2a0 0x7f998006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:10.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.810+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f998c089630 con 0x7f99940b5b40 2023-12-04T05:39:10.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.810+0000 7f99937fe640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f998006b2a0 0x7f998006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:10.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.810+0000 7f99937fe640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f998006b2a0 0x7f998006d760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f999415e9d0 tx=0x7f9984008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:10.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:10.814+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f998c054880 con 0x7f99940b5b40 2023-12-04T05:39:11.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.016+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9994067460 con 0x7f99940b5b40 2023-12-04T05:39:11.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.016+0000 7f997bfff640 1 -- 172.21.15.111:0/627281691 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f998c057ea0 con 0x7f99940b5b40 2023-12-04T05:39:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:11.018 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-04T05:39:11.018 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 2 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f998006b2a0 msgr2=0x7f998006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f998006b2a0 0x7f998006d760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f999415e9d0 tx=0x7f9984008040 comp rx=0 tx=0).stop 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 msgr2=0x7f9994159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 0x7f9994159610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b970 tx=0x7f998c00be30 comp rx=0 tx=0).stop 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 shutdown_connections 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f998006b2a0 0x7f998006d760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99940b5b40 0x7f9994159610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f999415edd0 0x7f999415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 --2- 172.21.15.111:0/627281691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999415b480 0x7f9994159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 >> 172.21.15.111:0/627281691 conn(0x7f99940b2a70 msgr2=0x7f999406e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 shutdown_connections 2023-12-04T05:39:11.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:11.018+0000 7f9999bd5640 1 -- 172.21.15.111:0/627281691 wait complete. 2023-12-04T05:39:13.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:13.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.141+0000 7f7261943640 1 Processor -- start 2023-12-04T05:39:13.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.141+0000 7f7261943640 1 -- start start 2023-12-04T05:39:13.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f7261943640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f725c0b20a0 0x7f725c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f7261943640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f725c0b2ad0 0x7f725c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f7261943640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0cdcb0 con 0x7f725c0d1780 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f7261943640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0d39e0 con 0x7f725c0b2ad0 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f7261943640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0d3cc0 con 0x7f725c0b20a0 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f725c0b2ad0 0x7f725c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725affd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f725c0d1780 0x7f725c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32830/0 (socket says 172.21.15.111:32830) 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725affd640 1 -- 172.21.15.111:0/105078825 learned_addr learned my addr 172.21.15.111:0/105078825 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f7253fff640 1 --2- 172.21.15.111:0/105078825 >> v2:172.21.15.192:3300/0 conn(0x7f725c0b20a0 0x7f725c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725b7fe640 1 -- 172.21.15.111:0/105078825 >> v2:172.21.15.192:3300/0 conn(0x7f725c0b20a0 msgr2=0x7f725c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725b7fe640 1 --2- 172.21.15.111:0/105078825 >> v2:172.21.15.192:3300/0 conn(0x7f725c0b20a0 0x7f725c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725b7fe640 1 -- 172.21.15.111:0/105078825 >> v1:172.21.15.111:6789/0 conn(0x7f725c0d1780 legacy=0x7f725c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725b7fe640 1 -- 172.21.15.111:0/105078825 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0cdf50 con 0x7f725c0b2ad0 2023-12-04T05:39:13.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.142+0000 7f725a7fc640 1 -- 172.21.15.111:0/105078825 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664505441 0 0) 0x7f725c0cdcb0 con 0x7f725c0d1780 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.143+0000 7f725b7fe640 1 --2- 172.21.15.111:0/105078825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b2ad0 0x7f725c0b2f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f725c162190 tx=0x7f725402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9176eb8166e5b8b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.143+0000 7f725a7fc640 1 -- 172.21.15.111:0/105078825 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725402cbf0 con 0x7f725c0b2ad0 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.143+0000 7f725a7fc640 1 -- 172.21.15.111:0/105078825 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725402cd50 con 0x7f725c0b2ad0 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.143+0000 7f725a7fc640 1 -- 172.21.15.111:0/105078825 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254034660 con 0x7f725c0b2ad0 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 -- 172.21.15.111:0/105078825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b2ad0 msgr2=0x7f725c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 --2- 172.21.15.111:0/105078825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b2ad0 0x7f725c0b2f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f725c162190 tx=0x7f725402be40 comp rx=0 tx=0).stop 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 -- 172.21.15.111:0/105078825 shutdown_connections 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 --2- 172.21.15.111:0/105078825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b2ad0 0x7f725c0b2f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 --2- 172.21.15.111:0/105078825 >> v2:172.21.15.192:3300/0 conn(0x7f725c0b20a0 0x7f725c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 -- 172.21.15.111:0/105078825 >> 172.21.15.111:0/105078825 conn(0x7f725c09c4e0 msgr2=0x7f725c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:13.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 -- 172.21.15.111:0/105078825 shutdown_connections 2023-12-04T05:39:13.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.144+0000 7f7261943640 1 -- 172.21.15.111:0/105078825 wait complete. 2023-12-04T05:39:13.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.145+0000 7f7261943640 1 Processor -- start 2023-12-04T05:39:13.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.145+0000 7f7261943640 1 -- start start 2023-12-04T05:39:13.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.145+0000 7f7261943640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 0x7f725c0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f725c0b2ad0 0x7f725c15ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f725c0d1780 0x7f725c14a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c1601a0 con 0x7f725c0b20a0 2023-12-04T05:39:13.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c15f590 con 0x7f725c0d1780 2023-12-04T05:39:13.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c15f870 con 0x7f725c0b2ad0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 0x7f725c0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 0x7f725c0ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47342/0 (socket says 172.21.15.111:47342) 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 -- 172.21.15.111:0/3574261468 learned_addr learned my addr 172.21.15.111:0/3574261468 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725b7fe640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f725c0d1780 0x7f725c14a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7253fff640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f725c0b2ad0 0x7f725c15ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 -- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f725c0b2ad0 msgr2=0x7f725c15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f725c0b2ad0 0x7f725c15ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 -- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f725c0d1780 msgr2=0x7f725c14a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f725c0d1780 0x7f725c14a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 -- 172.21.15.111:0/3574261468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254009a70 con 0x7f725c0b20a0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7253fff640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f725c0b2ad0 0x7f725c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f725affd640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 0x7f725c0ca4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c430 tx=0x7f724c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0074e0 con 0x7f725c0b20a0 2023-12-04T05:39:13.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c007660 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c15fb50 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c004a20 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.146+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c15a500 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.147+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c0b16a0 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.148+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f724c004b80 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.148+0000 7f7258ff9640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f723c06b2a0 0x7f723c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.148+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f724c0896a0 con 0x7f725c0b20a0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.148+0000 7f7253fff640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f723c06b2a0 0x7f723c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.148+0000 7f7253fff640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f723c06b2a0 0x7f723c06d760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f725c15a2b0 tx=0x7f7244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.152+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f724c0547d0 con 0x7f725c0b20a0 2023-12-04T05:39:13.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.355+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f725c0693d0 con 0x7f725c0b20a0 2023-12-04T05:39:13.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.355+0000 7f7258ff9640 1 -- 172.21.15.111:0/3574261468 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7f724c057df0 con 0x7f725c0b20a0 2023-12-04T05:39:13.357 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:13.357 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-04T05:39:13.357 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 3 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f723c06b2a0 msgr2=0x7f723c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f723c06b2a0 0x7f723c06d760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f725c15a2b0 tx=0x7f7244008040 comp rx=0 tx=0).stop 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 msgr2=0x7f725c0ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 0x7f725c0ca4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c430 tx=0x7f724c00c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 shutdown_connections 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f723c06b2a0 0x7f723c06d760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f725c0d1780 0x7f725c14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f725c0b2ad0 0x7f725c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 --2- 172.21.15.111:0/3574261468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f725c0b20a0 0x7f725c0ca4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 >> 172.21.15.111:0/3574261468 conn(0x7f725c09c4e0 msgr2=0x7f725c0d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:13.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 shutdown_connections 2023-12-04T05:39:13.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.357+0000 7f7261943640 1 -- 172.21.15.111:0/3574261468 wait complete. 2023-12-04T05:39:13.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:13.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.467+0000 7f8d8c3fc640 1 Processor -- start 2023-12-04T05:39:13.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.467+0000 7f8d8c3fc640 1 -- start start 2023-12-04T05:39:13.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.468+0000 7f8d8c3fc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8d8415b480 0x7f8d840b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.468+0000 7f8d8c3fc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840cbca0 con 0x7f8d8415f4e0 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.468+0000 7f8d8c3fc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840cbf40 con 0x7f8d840cf9f0 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.468+0000 7f8d8c3fc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840cc220 con 0x7f8d8415b480 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d8a999640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8d8415f4e0 0x7f8d840b08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32846/0 (socket says 172.21.15.111:32846) 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d8a999640 1 -- 172.21.15.111:0/3363341846 learned_addr learned my addr 172.21.15.111:0/3363341846 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d8a198640 1 --2- 172.21.15.111:0/3363341846 >> v2:172.21.15.192:3300/0 conn(0x7f8d8415b480 0x7f8d840b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892751878 0 0) 0x7f8d840cbca0 con 0x7f8d8415f4e0 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d6c003520 con 0x7f8d8415f4e0 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572530241 0 0) 0x7f8d6c003520 con 0x7f8d8415f4e0 2023-12-04T05:39:13.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 >> v2:172.21.15.192:3300/0 conn(0x7f8d8415b480 msgr2=0x7f8d840b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 --2- 172.21.15.111:0/3363341846 >> v2:172.21.15.192:3300/0 conn(0x7f8d8415b480 0x7f8d840b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 >> v1:172.21.15.192:6789/0 conn(0x7f8d840cf9f0 legacy=0x7f8d8415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840cc500 con 0x7f8d8415f4e0 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f8d7c001670 con 0x7f8d8415f4e0 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 >> v1:172.21.15.111:6789/0 conn(0x7f8d8415f4e0 legacy=0x7f8d840b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d6c003970 0x7f8d6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d6c005180 0x7f8d6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cc220 con 0x7f8d6c0047c0 2023-12-04T05:39:13.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c00c020 con 0x7f8d6c003970 2023-12-04T05:39:13.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c00c1e0 con 0x7f8d6c005180 2023-12-04T05:39:13.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89997640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d8a198640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d6c003970 0x7f8d6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d8a999640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d6c005180 0x7f8d6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89997640 1 -- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d6c005180 msgr2=0x7f8d6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:13.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89997640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d6c005180 0x7f8d6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89997640 1 -- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d6c003970 msgr2=0x7f8d6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:13.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89997640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d6c003970 0x7f8d6c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d89997640 1 -- 172.21.15.111:0/3363341846 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c003520 con 0x7f8d6c0047c0 2023-12-04T05:39:13.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.469+0000 7f8d8a999640 1 -- 172.21.15.111:0/3363341846 reap_dead start 2023-12-04T05:39:13.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.470+0000 7f8d89997640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d78001750 tx=0x7f8d6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f26f27c6f6a643 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.470+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780052d0 con 0x7f8d6c0047c0 2023-12-04T05:39:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.470+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7802e050 con 0x7f8d6c0047c0 2023-12-04T05:39:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.470+0000 7f8d89196640 1 -- 172.21.15.111:0/3363341846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78003830 con 0x7f8d6c0047c0 2023-12-04T05:39:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/3363341846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 msgr2=0x7f8d6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d78001750 tx=0x7f8d6c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/3363341846 shutdown_connections 2023-12-04T05:39:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/3363341846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/3363341846 >> 172.21.15.111:0/3363341846 conn(0x7f8d840b2a70 msgr2=0x7f8d840b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/3363341846 shutdown_connections 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.471+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/3363341846 wait complete. 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.472+0000 7f8d8c3fc640 1 Processor -- start 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.472+0000 7f8d8c3fc640 1 -- start start 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d8414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d840c5d90 0x7f8d840aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d840c6210 0x7f8d840af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840c73f0 con 0x7f8d6c0047c0 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840c67e0 con 0x7f8d840c6210 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840c6ac0 con 0x7f8d840c5d90 2023-12-04T05:39:13.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d8414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d8414fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47352/0 (socket says 172.21.15.111:47352) 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 -- 172.21.15.111:0/2478250352 learned_addr learned my addr 172.21.15.111:0/2478250352 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d89997640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d840c5d90 0x7f8d840aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 -- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d840c5d90 msgr2=0x7f8d840aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d840c5d90 0x7f8d840aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a999640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d840c6210 0x7f8d840af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 -- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d840c6210 msgr2=0x7f8d840af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d840c6210 0x7f8d840af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 -- 172.21.15.111:0/2478250352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840c6da0 con 0x7f8d6c0047c0 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d89997640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d840c5d90 0x7f8d840aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a999640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d840c6210 0x7f8d840af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:13.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8a198640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d8414fb00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009f00 tx=0x7f8d7400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400e040 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7400f040 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840c7080 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.473+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74003bb0 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.474+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d84150040 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.474+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d84067b70 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.475+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8d74003d10 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.475+0000 7f8d82ffd640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8d7006b2d0 0x7f8d7006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.475+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d7408a330 con 0x7f8d6c0047c0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.475+0000 7f8d89997640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8d7006b2d0 0x7f8d7006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.476+0000 7f8d89997640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8d7006b2d0 0x7f8d7006d790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d78033000 tx=0x7f8d78009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.479+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d74055580 con 0x7f8d6c0047c0 2023-12-04T05:39:13.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.646+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d84067460 con 0x7f8d6c0047c0 2023-12-04T05:39:13.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.647+0000 7f8d82ffd640 1 -- 172.21.15.111:0/2478250352 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f8d74058ba0 con 0x7f8d6c0047c0 2023-12-04T05:39:13.648 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:13.648 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:06.563565+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:13.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8d7006b2d0 msgr2=0x7f8d7006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8d7006b2d0 0x7f8d7006d790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d78033000 tx=0x7f8d78009030 comp rx=0 tx=0).stop 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 msgr2=0x7f8d8414fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d8414fb00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009f00 tx=0x7f8d7400f580 comp rx=0 tx=0).stop 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 shutdown_connections 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8d7006b2d0 0x7f8d7006d790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d840c6210 0x7f8d840af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d840c5d90 0x7f8d840aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 --2- 172.21.15.111:0/2478250352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c0047c0 0x7f8d8414fb00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 >> 172.21.15.111:0/2478250352 conn(0x7f8d840b2a70 msgr2=0x7f8d840b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 shutdown_connections 2023-12-04T05:39:13.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.649+0000 7f8d8c3fc640 1 -- 172.21.15.111:0/2478250352 wait complete. 2023-12-04T05:39:13.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:13.661 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:13.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.761+0000 7feec8c1e640 1 Processor -- start 2023-12-04T05:39:13.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.761+0000 7feec8c1e640 1 -- start start 2023-12-04T05:39:13.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec8c1e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feec40b3b20 0x7feec415da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec8c1e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4161400 con 0x7feec40d01d0 2023-12-04T05:39:13.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec8c1e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec415e280 con 0x7feec40b3740 2023-12-04T05:39:13.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec8c1e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec415e560 con 0x7feec40b3b20 2023-12-04T05:39:13.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec1d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feec40b3b20 0x7feec415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec2d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7feec40d01d0 0x7feec40b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32856/0 (socket says 172.21.15.111:32856) 2023-12-04T05:39:13.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.762+0000 7feec2d9d640 1 -- 172.21.15.111:0/2657546126 learned_addr learned my addr 172.21.15.111:0/2657546126 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:13.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.763+0000 7feec1d9b640 1 -- 172.21.15.111:0/2657546126 >> v1:172.21.15.192:6789/0 conn(0x7feec40b3740 legacy=0x7feec4161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.763+0000 7feec1d9b640 1 -- 172.21.15.111:0/2657546126 >> v1:172.21.15.111:6789/0 conn(0x7feec40d01d0 legacy=0x7feec40b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.763+0000 7feec1d9b640 1 -- 172.21.15.111:0/2657546126 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4161090 con 0x7feec40b3b20 2023-12-04T05:39:13.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.763+0000 7feec159a640 1 -- 172.21.15.111:0/2657546126 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237425122 0 0) 0x7feec4161400 con 0x7feec40d01d0 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.763+0000 7feec1d9b640 1 --2- 172.21.15.111:0/2657546126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40b3b20 0x7feec415da00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097a0 tx=0x7feeb802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332ae2e43c813b56 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.763+0000 7feec159a640 1 -- 172.21.15.111:0/2657546126 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8031830 con 0x7feec40b3b20 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec159a640 1 -- 172.21.15.111:0/2657546126 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8031990 con 0x7feec40b3b20 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec159a640 1 -- 172.21.15.111:0/2657546126 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8031d20 con 0x7feec40b3b20 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 -- 172.21.15.111:0/2657546126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40b3b20 msgr2=0x7feec415da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 --2- 172.21.15.111:0/2657546126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40b3b20 0x7feec415da00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097a0 tx=0x7feeb802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 -- 172.21.15.111:0/2657546126 shutdown_connections 2023-12-04T05:39:13.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 --2- 172.21.15.111:0/2657546126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40b3b20 0x7feec415da00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 -- 172.21.15.111:0/2657546126 >> 172.21.15.111:0/2657546126 conn(0x7feec40b5500 msgr2=0x7feec40b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:13.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 -- 172.21.15.111:0/2657546126 shutdown_connections 2023-12-04T05:39:13.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.764+0000 7feec8c1e640 1 -- 172.21.15.111:0/2657546126 wait complete. 2023-12-04T05:39:13.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.765+0000 7feec8c1e640 1 Processor -- start 2023-12-04T05:39:13.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.765+0000 7feec8c1e640 1 -- start start 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec8c1e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feec40b3740 0x7feec4141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec8c1e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feec40b3b20 0x7feec4158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec8c1e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 0x7feec415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec8c1e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4159600 con 0x7feec40d01d0 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec8c1e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4142240 con 0x7feec40b3740 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec8c1e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec41589f0 con 0x7feec40b3b20 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 0x7feec415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 0x7feec415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47370/0 (socket says 172.21.15.111:47370) 2023-12-04T05:39:13.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 -- 172.21.15.111:0/967038814 learned_addr learned my addr 172.21.15.111:0/967038814 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 -- 172.21.15.111:0/967038814 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feec40b3b20 msgr2=0x7feec4158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feec40b3b20 0x7feec4158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 -- 172.21.15.111:0/967038814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feec40b3740 msgr2=0x7feec4141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feec40b3740 0x7feec4141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.766+0000 7feec2d9d640 1 -- 172.21.15.111:0/967038814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8009400 con 0x7feec40d01d0 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.767+0000 7feec2d9d640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 0x7feec415faf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c450 tx=0x7feebc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.767+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0074e0 con 0x7feec40d01d0 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.767+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc007660 con 0x7feec40d01d0 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.767+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec4158cd0 con 0x7feec40d01d0 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.767+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc004a20 con 0x7feec40d01d0 2023-12-04T05:39:13.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.767+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec4159290 con 0x7feec40d01d0 2023-12-04T05:39:13.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.768+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4067460 con 0x7feec40d01d0 2023-12-04T05:39:13.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.768+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7feebc004b80 con 0x7feec40d01d0 2023-12-04T05:39:13.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.769+0000 7feeb2ffd640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feea406b2a0 0x7feea406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:13.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.769+0000 7feec259c640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feea406b2a0 0x7feea406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:13.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.769+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feebc088da0 con 0x7feec40d01d0 2023-12-04T05:39:13.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.769+0000 7feec259c640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feea406b2a0 0x7feea406d760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feeb40012e0 tx=0x7feeb4007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:13.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.773+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feebc054020 con 0x7feec40d01d0 2023-12-04T05:39:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.939+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feec40b32b0 con 0x7feec40d01d0 2023-12-04T05:39:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.939+0000 7feeb2ffd640 1 -- 172.21.15.111:0/967038814 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7feebc0180a0 con 0x7feec40d01d0 2023-12-04T05:39:13.941 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.941+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feea406b2a0 msgr2=0x7feea406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.941+0000 7feec8c1e640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feea406b2a0 0x7feea406d760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feeb40012e0 tx=0x7feeb4007d30 comp rx=0 tx=0).stop 2023-12-04T05:39:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.941+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 msgr2=0x7feec415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.941+0000 7feec8c1e640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 0x7feec415faf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c450 tx=0x7feebc00c910 comp rx=0 tx=0).stop 2023-12-04T05:39:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 shutdown_connections 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feea406b2a0 0x7feea406d760 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feec40d01d0 0x7feec415faf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feec40b3b20 0x7feec4158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 --2- 172.21.15.111:0/967038814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feec40b3740 0x7feec4141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 >> 172.21.15.111:0/967038814 conn(0x7feec40b5500 msgr2=0x7feec40b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 shutdown_connections 2023-12-04T05:39:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:13.942+0000 7feec8c1e640 1 -- 172.21.15.111:0/967038814 wait complete. 2023-12-04T05:39:13.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-04T05:39:14.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.051+0000 7fc1a9648640 1 Processor -- start 2023-12-04T05:39:14.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.051+0000 7fc1a9648640 1 -- start start 2023-12-04T05:39:14.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a9648640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc1a40ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a9648640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40cdcb0 con 0x7fc1a40d1780 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a9648640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40d39e0 con 0x7fc1a40b20a0 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a9648640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40d0610 con 0x7fc1a40d01e0 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a2ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc1a40d1780 0x7fc1a40d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32870/0 (socket says 172.21.15.111:32870) 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a2ffd640 1 -- 172.21.15.111:0/961764689 learned_addr learned my addr 172.21.15.111:0/961764689 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068221733 0 0) 0x7fc1a40cdcb0 con 0x7fc1a40d1780 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a37fe640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc1a40ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc18c003520 con 0x7fc1a40d1780 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268586223 0 0) 0x7fc18c003520 con 0x7fc1a40d1780 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 msgr2=0x7fc1a40ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc1a40ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 >> v1:172.21.15.192:6789/0 conn(0x7fc1a40b20a0 legacy=0x7fc1a40b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40d3cc0 con 0x7fc1a40d1780 2023-12-04T05:39:14.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fc19c001670 con 0x7fc1a40d1780 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 >> v1:172.21.15.111:6789/0 conn(0x7fc1a40d1780 legacy=0x7fc1a40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0039a0 0x7fc18c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc18c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc18c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40d0610 con 0x7fc18c004d00 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c00bab0 con 0x7fc18c0039a0 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c00bc70 con 0x7fc1a40d01e0 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a37fe640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc18c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a2ffd640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0039a0 0x7fc18c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.052+0000 7fc1a27fc640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc18c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a37fe640 1 -- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 msgr2=0x7fc18c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a37fe640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc18c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a37fe640 1 -- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0039a0 msgr2=0x7fc18c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:14.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a37fe640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0039a0 0x7fc18c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a37fe640 1 -- 172.21.15.111:0/961764689 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c003520 con 0x7fc18c004d00 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a27fc640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40d01e0 0x7fc18c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a37fe640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc18c005180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc194009ac0 tx=0x7fc18c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17647e93afd2ac6e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19402c6b0 con 0x7fc18c004d00 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.053+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19402c810 con 0x7fc18c004d00 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a1ffb640 1 -- 172.21.15.111:0/961764689 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19402cba0 con 0x7fc18c004d00 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a9648640 1 -- 172.21.15.111:0/961764689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 msgr2=0x7fc18c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a9648640 1 --2- 172.21.15.111:0/961764689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc18c005180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc194009ac0 tx=0x7fc18c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a37fe640 1 -- 172.21.15.111:0/961764689 reap_dead start 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a9648640 1 -- 172.21.15.111:0/961764689 shutdown_connections 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a9648640 1 -- 172.21.15.111:0/961764689 >> 172.21.15.111:0/961764689 conn(0x7fc1a409c4e0 msgr2=0x7fc1a40d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a9648640 1 -- 172.21.15.111:0/961764689 shutdown_connections 2023-12-04T05:39:14.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.054+0000 7fc1a9648640 1 -- 172.21.15.111:0/961764689 wait complete. 2023-12-04T05:39:14.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.055+0000 7fc1a9648640 1 Processor -- start 2023-12-04T05:39:14.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.055+0000 7fc1a9648640 1 -- start start 2023-12-04T05:39:14.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a9648640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc1a40c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a9648640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40c6c50 0x7fc1a41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a9648640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a40cc760 0x7fc1a40ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a9648640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a4155d30 con 0x7fc18c004d00 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a9648640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40ca270 con 0x7fc1a40cc760 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a9648640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40cd120 con 0x7fc1a40c6c50 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc1a40c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc1a40c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47390/0 (socket says 172.21.15.111:47390) 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 -- 172.21.15.111:0/716335310 learned_addr learned my addr 172.21.15.111:0/716335310 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a37fe640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a40cc760 0x7fc1a40ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a27fc640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40c6c50 0x7fc1a41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 -- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40c6c50 msgr2=0x7fc1a41557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40c6c50 0x7fc1a41557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 -- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a40cc760 msgr2=0x7fc1a40ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a40cc760 0x7fc1a40ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.056+0000 7fc1a2ffd640 1 -- 172.21.15.111:0/716335310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40c8150 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.057+0000 7fc1a2ffd640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc1a40c7b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0072c0 tx=0x7fc19c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.057+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00f960 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.057+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00fac0 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.057+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a40c8430 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.057+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00fe70 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.057+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a40ca7a0 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.058+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a40c85f0 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.058+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc19c01d460 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.058+0000 7fc18b7fe640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc19006b2d0 0x7fc19006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.058+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc19c089990 con 0x7fc18c004d00 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.058+0000 7fc1a27fc640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc19006b2d0 0x7fc19006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.059+0000 7fc1a27fc640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc19006b2d0 0x7fc19006d790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc198007fd0 tx=0x7fc198009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:14.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.063+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc19c054be0 con 0x7fc18c004d00 2023-12-04T05:39:14.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.232+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc1a40693d0 con 0x7fc18c004d00 2023-12-04T05:39:14.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.232+0000 7fc18b7fe640 1 -- 172.21.15.111:0/716335310 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc19c058200 con 0x7fc18c004d00 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:39:14.234 INFO:teuthology.orchestra.run.smithi111.stdout:] 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc19006b2d0 msgr2=0x7fc19006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc19006b2d0 0x7fc19006d790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc198007fd0 tx=0x7fc198009040 comp rx=0 tx=0).stop 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 msgr2=0x7fc1a40c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc1a40c7b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0072c0 tx=0x7fc19c007780 comp rx=0 tx=0).stop 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 shutdown_connections 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc19006b2d0 0x7fc19006d790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a40cc760 0x7fc1a40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a40c6c50 0x7fc1a41557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 --2- 172.21.15.111:0/716335310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c004d00 0x7fc1a40c7b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 >> 172.21.15.111:0/716335310 conn(0x7fc1a409c4e0 msgr2=0x7fc1a40cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 shutdown_connections 2023-12-04T05:39:14.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.234+0000 7fc1a9648640 1 -- 172.21.15.111:0/716335310 wait complete. 2023-12-04T05:39:14.246 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-04T05:39:14.247 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-04T05:39:14.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.348+0000 7f0c0017c640 1 Processor -- start 2023-12-04T05:39:14.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.349+0000 7f0c0017c640 1 -- start start 2023-12-04T05:39:14.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.349+0000 7f0c0017c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0bf8146b10 0x7f0bf8161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.349+0000 7f0c0017c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0bf80d0f10 0x7f0bf80d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.349+0000 7f0c0017c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf806eaf0 con 0x7f0bf815d8a0 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.349+0000 7f0c0017c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf80d18e0 con 0x7f0bf8146b10 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.349+0000 7f0c0017c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf815e4e0 con 0x7f0bf80d0f10 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0bf8146b10 0x7f0bf8161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0bf8146b10 0x7f0bf8161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47402/0 (socket says 172.21.15.111:47402) 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 -- 172.21.15.111:0/4104346375 learned_addr learned my addr 172.21.15.111:0/4104346375 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfd717640 1 --2- 172.21.15.111:0/4104346375 >> v2:172.21.15.192:3300/0 conn(0x7f0bf80d0f10 0x7f0bf80d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 -- 172.21.15.111:0/4104346375 >> v2:172.21.15.192:3300/0 conn(0x7f0bf80d0f10 msgr2=0x7f0bf80d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 --2- 172.21.15.111:0/4104346375 >> v2:172.21.15.192:3300/0 conn(0x7f0bf80d0f10 0x7f0bf80d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 -- 172.21.15.111:0/4104346375 >> v1:172.21.15.192:6789/0 conn(0x7f0bf815d8a0 legacy=0x7f0bf815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfdf18640 1 -- 172.21.15.111:0/4104346375 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf815e7c0 con 0x7f0bf8146b10 2023-12-04T05:39:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.350+0000 7f0bfd717640 1 --2- 172.21.15.111:0/4104346375 >> v2:172.21.15.192:3300/0 conn(0x7f0bf80d0f10 0x7f0bf80d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0bfdf18640 1 --2- 172.21.15.111:0/4104346375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8146b10 0x7f0bf8161a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80b6750 tx=0x7f0bf402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a606a0ec6245ffb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0bfcf16640 1 -- 172.21.15.111:0/4104346375 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4031870 con 0x7f0bf8146b10 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0bfcf16640 1 -- 172.21.15.111:0/4104346375 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf40319d0 con 0x7f0bf8146b10 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0bfcf16640 1 -- 172.21.15.111:0/4104346375 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4031d60 con 0x7f0bf8146b10 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 -- 172.21.15.111:0/4104346375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8146b10 msgr2=0x7f0bf8161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 --2- 172.21.15.111:0/4104346375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8146b10 0x7f0bf8161a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80b6750 tx=0x7f0bf402b910 comp rx=0 tx=0).stop 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 -- 172.21.15.111:0/4104346375 shutdown_connections 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 --2- 172.21.15.111:0/4104346375 >> v2:172.21.15.192:3300/0 conn(0x7f0bf80d0f10 0x7f0bf80d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 --2- 172.21.15.111:0/4104346375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8146b10 0x7f0bf8161a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 -- 172.21.15.111:0/4104346375 >> 172.21.15.111:0/4104346375 conn(0x7f0bf8070af0 msgr2=0x7f0bf80662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 -- 172.21.15.111:0/4104346375 shutdown_connections 2023-12-04T05:39:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.351+0000 7f0c0017c640 1 -- 172.21.15.111:0/4104346375 wait complete. 2023-12-04T05:39:14.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.352+0000 7f0c0017c640 1 Processor -- start 2023-12-04T05:39:14.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.352+0000 7f0c0017c640 1 -- start start 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0c0017c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf80d0f10 0x7f0bf80afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0c0017c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0bf8146b10 0x7f0bf80cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0c0017c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 0x7f0bf80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0c0017c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81317c0 con 0x7f0bf815d8a0 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0c0017c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf80cb3b0 con 0x7f0bf80d0f10 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0c0017c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf80cb690 con 0x7f0bf8146b10 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0bfe719640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 0x7f0bf80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0bfe719640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 0x7f0bf80cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47418/0 (socket says 172.21.15.111:47418) 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0bfe719640 1 -- 172.21.15.111:0/1702212398 learned_addr learned my addr 172.21.15.111:0/1702212398 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0bfd717640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0bf8146b10 0x7f0bf80cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.353+0000 7f0bfdf18640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf80d0f10 0x7f0bf80afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bfe719640 1 -- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0bf8146b10 msgr2=0x7f0bf80cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bfe719640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0bf8146b10 0x7f0bf80cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bfe719640 1 -- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf80d0f10 msgr2=0x7f0bf80afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bfe719640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf80d0f10 0x7f0bf80afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bfe719640 1 -- 172.21.15.111:0/1702212398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40094d0 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bfe719640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 0x7f0bf80cf380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b970 tx=0x7f0bf000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000cb40 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf000cca0 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0014680 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf80cb970 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.354+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf80b0260 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.355+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf8131970 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.355+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0bf001d3b0 con 0x7f0bf815d8a0 2023-12-04T05:39:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.356+0000 7f0bee7fc640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0bdc06b2a0 0x7f0bdc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.356+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0bf0089630 con 0x7f0bf815d8a0 2023-12-04T05:39:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.356+0000 7f0bfdf18640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0bdc06b2a0 0x7f0bdc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.356+0000 7f0bfdf18640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0bdc06b2a0 0x7f0bdc06d760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4036000 tx=0x7f0bf403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:14.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.360+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bf8131970 con 0x7f0bf815d8a0 2023-12-04T05:39:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:14.530+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0bf80b5130 con 0x7f0bf815d8a0 2023-12-04T05:39:15.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.196+0000 7f0bee7fc640 1 -- 172.21.15.111:0/1702212398 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0bf0054880 con 0x7f0bf815d8a0 2023-12-04T05:39:15.198 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2023-12-04T05:39:15.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0bdc06b2a0 msgr2=0x7f0bdc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:15.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0bdc06b2a0 0x7f0bdc06d760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4036000 tx=0x7f0bf403c040 comp rx=0 tx=0).stop 2023-12-04T05:39:15.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 msgr2=0x7f0bf80cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 0x7f0bf80cf380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b970 tx=0x7f0bf000be30 comp rx=0 tx=0).stop 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 shutdown_connections 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0bdc06b2a0 0x7f0bdc06d760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf815d8a0 0x7f0bf80cf380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0bf8146b10 0x7f0bf80cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 --2- 172.21.15.111:0/1702212398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf80d0f10 0x7f0bf80afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 >> 172.21.15.111:0/1702212398 conn(0x7f0bf8070af0 msgr2=0x7f0bf815f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 shutdown_connections 2023-12-04T05:39:15.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.199+0000 7f0c0017c640 1 -- 172.21.15.111:0/1702212398 wait complete. 2023-12-04T05:39:15.215 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-04T05:39:15.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.321+0000 7f1a04da0640 1 Processor -- start 2023-12-04T05:39:15.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.321+0000 7f1a04da0640 1 -- start start 2023-12-04T05:39:15.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f1a04da0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a000d1460 0x7f1a000b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f1a04da0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 0x7f1a000adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f1a04da0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a000b1960 con 0x7f1a0015afa0 2023-12-04T05:39:15.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f1a04da0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000b1c40 con 0x7f1a000d1460 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f1a04da0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b1f20 con 0x7f1a000b11b0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f19ff7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1a0015afa0 0x7f1a000d0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32876/0 (socket says 172.21.15.111:32876) 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.322+0000 7f19ff7fe640 1 -- 172.21.15.111:0/3589195756 learned_addr learned my addr 172.21.15.111:0/3589195756 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874089759 0 0) 0x7f1a000b1960 con 0x7f1a0015afa0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19feffd640 1 --2- 172.21.15.111:0/3589195756 >> v2:172.21.15.192:3300/0 conn(0x7f1a000d1460 0x7f1a000b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e8003520 con 0x7f1a0015afa0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 0x7f1a000adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898389580 0 0) 0x7f19e8003520 con 0x7f1a0015afa0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 msgr2=0x7f1a000adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 0x7f1a000adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 >> v2:172.21.15.192:3300/0 conn(0x7f1a000d1460 msgr2=0x7f1a000b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 --2- 172.21.15.111:0/3589195756 >> v2:172.21.15.192:3300/0 conn(0x7f1a000d1460 0x7f1a000b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000b2200 con 0x7f1a0015afa0 2023-12-04T05:39:15.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f19f0001670 con 0x7f1a0015afa0 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 >> v1:172.21.15.111:6789/0 conn(0x7f1a0015afa0 legacy=0x7f1a000d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 0x7f19e8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e8003ef0 0x7f19e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e8004d00 0x7f19e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b1c40 con 0x7f19e8004d00 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000b1f20 con 0x7f19e8003ef0 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e800b980 con 0x7f1a000b11b0 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e8004d00 0x7f19e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19ff7fe640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 0x7f19e8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19feffd640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e8003ef0 0x7f19e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 -- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 msgr2=0x7f19e8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a000b11b0 0x7f19e8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 -- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e8003ef0 msgr2=0x7f19e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:15.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19e8003ef0 0x7f19e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.323+0000 7f19fffff640 1 -- 172.21.15.111:0/3589195756 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e8003520 con 0x7f19e8004d00 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.324+0000 7f19fffff640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e8004d00 0x7f19e8005180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009fc0 tx=0x7f19e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c319330e4aa42d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.324+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f802cc00 con 0x7f19e8004d00 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.324+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800f040 con 0x7f19e8004d00 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.324+0000 7f19fe7fc640 1 -- 172.21.15.111:0/3589195756 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8034440 con 0x7f19e8004d00 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f1a04da0640 1 -- 172.21.15.111:0/3589195756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e8004d00 msgr2=0x7f19e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3589195756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e8004d00 0x7f19e8005180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009fc0 tx=0x7f19e80057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f19fffff640 1 -- 172.21.15.111:0/3589195756 reap_dead start 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f1a04da0640 1 -- 172.21.15.111:0/3589195756 shutdown_connections 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f1a04da0640 1 -- 172.21.15.111:0/3589195756 >> 172.21.15.111:0/3589195756 conn(0x7f1a00099a00 msgr2=0x7f1a00099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f1a04da0640 1 -- 172.21.15.111:0/3589195756 shutdown_connections 2023-12-04T05:39:15.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.325+0000 7f1a04da0640 1 -- 172.21.15.111:0/3589195756 wait complete. 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.326+0000 7f1a04da0640 1 Processor -- start 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.326+0000 7f1a04da0640 1 -- start start 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f1a04da0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a001f00a0 0x7f1a000aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f1a04da0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a000aefc0 0x7f1a000c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f1a04da0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 0x7f1a0006b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f1a04da0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000c9af0 con 0x7f1a000c84c0 2023-12-04T05:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f1a04da0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000af440 con 0x7f1a000aefc0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f1a04da0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000c97a0 con 0x7f1a001f00a0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 0x7f1a0006b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 0x7f1a0006b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47440/0 (socket says 172.21.15.111:47440) 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 -- 172.21.15.111:0/3315157930 learned_addr learned my addr 172.21.15.111:0/3315157930 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19feffd640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a000aefc0 0x7f1a000c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19ff7fe640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a001f00a0 0x7f1a000aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 -- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a001f00a0 msgr2=0x7f1a000aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a001f00a0 0x7f1a000aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 -- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a000aefc0 msgr2=0x7f1a000c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a000aefc0 0x7f1a000c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 -- 172.21.15.111:0/3315157930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000c8c70 con 0x7f1a000c84c0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19feffd640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a000aefc0 0x7f1a000c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19ff7fe640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a001f00a0 0x7f1a000aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.327+0000 7f19fffff640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 0x7f1a0006b5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19f8037040 tx=0x7f19f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:15.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.328+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8034d50 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.328+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f802de90 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.328+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a000c8f50 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.328+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8003bd0 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.328+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a000c8940 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.329+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a000c9c30 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.329+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f19f8003d30 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.329+0000 7f19e7fff640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f19ec06b2a0 0x7f19ec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.330+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f19f80ab3a0 con 0x7f1a000c84c0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.330+0000 7f19ff7fe640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f19ec06b2a0 0x7f19ec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.330+0000 7f19ff7fe640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f19ec06b2a0 0x7f19ec06d760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f19f0001050 tx=0x7f19f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.334+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a000c9c30 con 0x7f1a000c84c0 2023-12-04T05:39:15.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:15.503+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a00067460 con 0x7f1a000c84c0 2023-12-04T05:39:16.121 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:39:16.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:39:16.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.199+0000 7f19e7fff640 1 -- 172.21.15.111:0/3315157930 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f19f80764a0 con 0x7f1a000c84c0 2023-12-04T05:39:16.200 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f19ec06b2a0 msgr2=0x7f19ec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f19ec06b2a0 0x7f19ec06d760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f19f0001050 tx=0x7f19f0009040 comp rx=0 tx=0).stop 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 msgr2=0x7f1a0006b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 0x7f1a0006b5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19f8037040 tx=0x7f19f8009750 comp rx=0 tx=0).stop 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 shutdown_connections 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f19ec06b2a0 0x7f19ec06d760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a000c84c0 0x7f1a0006b5d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a000aefc0 0x7f1a000c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 --2- 172.21.15.111:0/3315157930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a001f00a0 0x7f1a000aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.201+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 >> 172.21.15.111:0/3315157930 conn(0x7f1a00099a00 msgr2=0x7f1a000adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:16.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.202+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 shutdown_connections 2023-12-04T05:39:16.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.202+0000 7f1a04da0640 1 -- 172.21.15.111:0/3315157930 wait complete. 2023-12-04T05:39:16.225 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-04T05:39:16.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.327+0000 7fbc73589640 1 Processor -- start 2023-12-04T05:39:16.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.327+0000 7fbc73589640 1 -- start start 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc73589640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbc6c13f640 0x7fbc6c13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc73589640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c071d20 0x7fbc6c072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc73589640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c13dd10 con 0x7fbc6c13cee0 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc73589640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c072670 con 0x7fbc6c13f640 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc73589640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c13d2c0 con 0x7fbc6c071d20 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc70b24640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbc6c13f640 0x7fbc6c13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc71b26640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c071d20 0x7fbc6c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:16.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc71325640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fbc6c13cee0 0x7fbc6c13d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:37834/0 (socket says 172.21.15.111:37834) 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc70b24640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbc6c13f640 0x7fbc6c13fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50288/0 (socket says 172.21.15.111:50288) 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.328+0000 7fbc71325640 1 -- 172.21.15.111:0/1727591204 learned_addr learned my addr 172.21.15.111:0/1727591204 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.329+0000 7fbc70b24640 1 -- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c071d20 msgr2=0x7fbc6c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.329+0000 7fbc70b24640 1 --2- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c071d20 0x7fbc6c072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.329+0000 7fbc70b24640 1 -- 172.21.15.111:0/1727591204 >> v1:172.21.15.192:6789/0 conn(0x7fbc6c13cee0 legacy=0x7fbc6c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.329+0000 7fbc70b24640 1 -- 172.21.15.111:0/1727591204 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c1400e0 con 0x7fbc6c13f640 2023-12-04T05:39:16.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.329+0000 7fbc71b26640 1 --2- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c071d20 0x7fbc6c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:16.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.329+0000 7fbc70b24640 1 --2- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c13fab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0097a0 tx=0x7fbc5c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30add2f28531c6d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:16.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc63fff640 1 -- 172.21.15.111:0/1727591204 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c02f050 con 0x7fbc6c13f640 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc63fff640 1 -- 172.21.15.111:0/1727591204 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c031700 con 0x7fbc6c13f640 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc63fff640 1 -- 172.21.15.111:0/1727591204 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c031a90 con 0x7fbc6c13f640 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 -- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 msgr2=0x7fbc6c13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 --2- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c13fab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0097a0 tx=0x7fbc5c02b7a0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 -- 172.21.15.111:0/1727591204 shutdown_connections 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 --2- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c071d20 0x7fbc6c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 --2- 172.21.15.111:0/1727591204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c13fab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 -- 172.21.15.111:0/1727591204 >> 172.21.15.111:0/1727591204 conn(0x7fbc6c0b0cf0 msgr2=0x7fbc6c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 -- 172.21.15.111:0/1727591204 shutdown_connections 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.330+0000 7fbc73589640 1 -- 172.21.15.111:0/1727591204 wait complete. 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.331+0000 7fbc73589640 1 Processor -- start 2023-12-04T05:39:16.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.331+0000 7fbc73589640 1 -- start start 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc73589640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 0x7fbc6c136620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc73589640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c13cee0 0x7fbc6c136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc73589640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c139330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc73589640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c141760 con 0x7fbc6c071d20 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc73589640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c13a140 con 0x7fbc6c13f640 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc73589640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c141130 con 0x7fbc6c13cee0 2023-12-04T05:39:16.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 0x7fbc6c136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 0x7fbc6c136620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47448/0 (socket says 172.21.15.111:47448) 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 -- 172.21.15.111:0/2164807039 learned_addr learned my addr 172.21.15.111:0/2164807039 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71b26640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc70b24640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c13cee0 0x7fbc6c136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 -- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c13cee0 msgr2=0x7fbc6c136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c13cee0 0x7fbc6c136070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 -- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 msgr2=0x7fbc6c139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.332+0000 7fbc71325640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc71325640 1 -- 172.21.15.111:0/2164807039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc64009400 con 0x7fbc6c071d20 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc71b26640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc70b24640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c13cee0 0x7fbc6c136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc71325640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 0x7fbc6c136620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800cba0 tx=0x7fbc6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:16.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800ad40 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68012400 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c009400 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800aea0 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.333+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c139e60 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.334+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c0cddf0 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.334+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbc68007070 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.335+0000 7fbc61ffb640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc5006b2d0 0x7fbc5006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.335+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbc68089a60 con 0x7fbc6c071d20 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.335+0000 7fbc70b24640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc5006b2d0 0x7fbc5006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.335+0000 7fbc70b24640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc5006b2d0 0x7fbc5006d790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c035000 tx=0x7fbc5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:16.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.339+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc680549f0 con 0x7fbc6c071d20 2023-12-04T05:39:16.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:16.541+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc6c0cdb10 con 0x7fbc6c071d20 2023-12-04T05:39:17.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.217+0000 7fbc61ffb640 1 -- 172.21.15.111:0/2164807039 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbc68058010 con 0x7fbc6c071d20 2023-12-04T05:39:17.219 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 2 and data pool 3 2023-12-04T05:39:17.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc5006b2d0 msgr2=0x7fbc5006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc5006b2d0 0x7fbc5006d790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c035000 tx=0x7fbc5c008030 comp rx=0 tx=0).stop 2023-12-04T05:39:17.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 msgr2=0x7fbc6c136620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 0x7fbc6c136620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800cba0 tx=0x7fbc6800a430 comp rx=0 tx=0).stop 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 shutdown_connections 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc5006b2d0 0x7fbc5006d790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6c13f640 0x7fbc6c139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6c13cee0 0x7fbc6c136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 --2- 172.21.15.111:0/2164807039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc6c071d20 0x7fbc6c136620 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 >> 172.21.15.111:0/2164807039 conn(0x7fbc6c0b0cf0 msgr2=0x7fbc6c0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 shutdown_connections 2023-12-04T05:39:17.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.220+0000 7fbc73589640 1 -- 172.21.15.111:0/2164807039 wait complete. 2023-12-04T05:39:17.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:17.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.380+0000 7fc6e2c83640 1 Processor -- start 2023-12-04T05:39:17.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.380+0000 7fc6e2c83640 1 -- start start 2023-12-04T05:39:17.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.381+0000 7fc6e2c83640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc6dc15dab0 0x7fc6dc0703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.381+0000 7fc6e2c83640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc154220 con 0x7fc6dc0b3310 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.381+0000 7fc6e2c83640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc15d2f0 con 0x7fc6dc06cae0 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.381+0000 7fc6e2c83640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc15a3c0 con 0x7fc6dc15dab0 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.381+0000 7fc6dbfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc6dc0b3310 0x7fc6dc159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32878/0 (socket says 172.21.15.111:32878) 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.381+0000 7fc6dbfff640 1 -- 172.21.15.111:0/27418413 learned_addr learned my addr 172.21.15.111:0/27418413 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6e1220640 1 --2- 172.21.15.111:0/27418413 >> v2:172.21.15.192:3300/0 conn(0x7fc6dc15dab0 0x7fc6dc0703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565621922 0 0) 0x7fc6d0001110 con 0x7fc6dc0b3310 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c4005730 con 0x7fc6dc0b3310 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3615175019 0 0) 0x7fc6c4005730 con 0x7fc6dc0b3310 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 >> v2:172.21.15.192:3300/0 conn(0x7fc6dc15dab0 msgr2=0x7fc6dc0703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 --2- 172.21.15.111:0/27418413 >> v2:172.21.15.192:3300/0 conn(0x7fc6dc15dab0 0x7fc6dc0703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 >> v1:172.21.15.192:6789/0 conn(0x7fc6dc06cae0 legacy=0x7fc6dc06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc15a5c0 con 0x7fc6dc0b3310 2023-12-04T05:39:17.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fc6d0001800 con 0x7fc6dc0b3310 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 >> v1:172.21.15.111:6789/0 conn(0x7fc6dc0b3310 legacy=0x7fc6dc159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6c4005b80 0x7fc6c4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6c40069d0 0x7fc6c4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c4007390 0x7fc6c400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc15a3c0 con 0x7fc6c40069d0 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400e230 con 0x7fc6c4007390 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c400e3f0 con 0x7fc6c4005b80 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6c40069d0 0x7fc6c4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6e0a1f640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6c4005b80 0x7fc6c4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6e1220640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c4007390 0x7fc6c400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6dbfff640 1 -- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6c4005b80 msgr2=0x7fc6c4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6c4005b80 0x7fc6c4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6dbfff640 1 -- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c4007390 msgr2=0x7fc6c400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:17.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c4007390 0x7fc6c400db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6dbfff640 1 -- 172.21.15.111:0/27418413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4005730 con 0x7fc6c40069d0 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6e1220640 1 -- 172.21.15.111:0/27418413 reap_dead start 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.382+0000 7fc6e0a1f640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6c4005b80 0x7fc6c4005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.383+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6c40069d0 0x7fc6c4006e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c160 tx=0x7fc6c4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0ebbd9c6645074 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.383+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0005ad0 con 0x7fc6c40069d0 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.383+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d0005c50 con 0x7fc6c40069d0 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.383+0000 7fc6db7fe640 1 -- 172.21.15.111:0/27418413 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0007070 con 0x7fc6c40069d0 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 -- 172.21.15.111:0/27418413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6c40069d0 msgr2=0x7fc6c4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6c40069d0 0x7fc6c4006e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c160 tx=0x7fc6c4007a00 comp rx=0 tx=0).stop 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 -- 172.21.15.111:0/27418413 shutdown_connections 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/27418413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6c40069d0 0x7fc6c4006e50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 -- 172.21.15.111:0/27418413 >> 172.21.15.111:0/27418413 conn(0x7fc6dc06a4c0 msgr2=0x7fc6dc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:17.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 -- 172.21.15.111:0/27418413 shutdown_connections 2023-12-04T05:39:17.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.384+0000 7fc6e2c83640 1 -- 172.21.15.111:0/27418413 wait complete. 2023-12-04T05:39:17.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.385+0000 7fc6e2c83640 1 Processor -- start 2023-12-04T05:39:17.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.385+0000 7fc6e2c83640 1 -- start start 2023-12-04T05:39:17.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c40069d0 0x7fc6dc145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 0x7fc6dc0ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6dc158040 0x7fc6dc1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc149d70 con 0x7fc6dc0ad280 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc158a30 con 0x7fc6c40069d0 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc145ff0 con 0x7fc6dc158040 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 0x7fc6dc0ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 0x7fc6dc0ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47466/0 (socket says 172.21.15.111:47466) 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 -- 172.21.15.111:0/2599458049 learned_addr learned my addr 172.21.15.111:0/2599458049 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e0a1f640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c40069d0 0x7fc6dc145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e1220640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6dc158040 0x7fc6dc1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 -- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6dc158040 msgr2=0x7fc6dc1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6dc158040 0x7fc6dc1584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 -- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c40069d0 msgr2=0x7fc6dc145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c40069d0 0x7fc6dc145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 -- 172.21.15.111:0/2599458049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc1496d0 con 0x7fc6dc0ad280 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6dbfff640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 0x7fc6dc0ad6e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0001ac0 tx=0x7fc6dc0716e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0039030 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d0007480 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.386+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc14bfb0 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.387+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d003c040 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.387+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc0ae0b0 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.387+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc149eb0 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.388+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc6d0032070 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.388+0000 7fc6d97fa640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc6c806b2d0 0x7fc6c806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.388+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6d00aad20 con 0x7fc6dc0ad280 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.388+0000 7fc6e0a1f640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc6c806b2d0 0x7fc6c806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.389+0000 7fc6e0a1f640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc6c806b2d0 0x7fc6c806d790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc000c00 tx=0x7fc6cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.392+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6dc149eb0 con 0x7fc6dc0ad280 2023-12-04T05:39:17.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.559+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6dc154220 con 0x7fc6dc0ad280 2023-12-04T05:39:17.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.560+0000 7fc6d97fa640 1 -- 172.21.15.111:0/2599458049 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16494 (secure 0 0 0) 0x7fc6dc154220 con 0x7fc6dc0ad280 2023-12-04T05:39:17.562 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:17.562 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":19,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:16.542580+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc6c806b2d0 msgr2=0x7fc6c806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc6c806b2d0 0x7fc6c806d790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc000c00 tx=0x7fc6cc009040 comp rx=0 tx=0).stop 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 msgr2=0x7fc6dc0ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 0x7fc6dc0ad6e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0001ac0 tx=0x7fc6dc0716e0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 shutdown_connections 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc6c806b2d0 0x7fc6c806d790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6dc158040 0x7fc6dc1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6dc0ad280 0x7fc6dc0ad6e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 --2- 172.21.15.111:0/2599458049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6c40069d0 0x7fc6dc145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 >> 172.21.15.111:0/2599458049 conn(0x7fc6dc06a4c0 msgr2=0x7fc6dc0b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:17.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 shutdown_connections 2023-12-04T05:39:17.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.562+0000 7fc6e2c83640 1 -- 172.21.15.111:0/2599458049 wait complete. 2023-12-04T05:39:17.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:17.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.673+0000 7f36f1a94640 1 Processor -- start 2023-12-04T05:39:17.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.673+0000 7f36f1a94640 1 -- start start 2023-12-04T05:39:17.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36f1a94640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec146a50 0x7f36ec161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36f1a94640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f36ec0cd970 0x7f36ec0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36f1a94640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0cadf0 con 0x7f36ec0b2ba0 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36f1a94640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec0ce7c0 con 0x7f36ec0cd970 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36f1a94640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0ce4e0 con 0x7f36ec146a50 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36eaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f36ec0cd970 0x7f36ec0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36ebfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f36ec0b2ba0 0x7f36ec06be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32888/0 (socket says 172.21.15.111:32888) 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36ebfff640 1 -- 172.21.15.111:0/2668664201 learned_addr learned my addr 172.21.15.111:0/2668664201 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36eb7fe640 1 --2- 172.21.15.111:0/2668664201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec146a50 0x7f36ec161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36eaffd640 1 -- 172.21.15.111:0/2668664201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec146a50 msgr2=0x7f36ec161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36eaffd640 1 --2- 172.21.15.111:0/2668664201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec146a50 0x7f36ec161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36eaffd640 1 -- 172.21.15.111:0/2668664201 >> v1:172.21.15.111:6789/0 conn(0x7f36ec0b2ba0 legacy=0x7f36ec06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36ea7fc640 1 -- 172.21.15.111:0/2668664201 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293114408 0 0) 0x7f36ec0cadf0 con 0x7f36ec0b2ba0 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.674+0000 7f36eaffd640 1 -- 172.21.15.111:0/2668664201 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec160fa0 con 0x7f36ec0cd970 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.675+0000 7f36eaffd640 1 --2- 172.21.15.111:0/2668664201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0cddd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f36e00097a0 tx=0x7f36e002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58026ba62dcab281 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.675+0000 7f36ea7fc640 1 -- 172.21.15.111:0/2668664201 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e002f050 con 0x7f36ec0cd970 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.675+0000 7f36ea7fc640 1 -- 172.21.15.111:0/2668664201 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0031700 con 0x7f36ec0cd970 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.675+0000 7f36ea7fc640 1 -- 172.21.15.111:0/2668664201 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0031a90 con 0x7f36ec0cd970 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 -- 172.21.15.111:0/2668664201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 msgr2=0x7f36ec0cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 --2- 172.21.15.111:0/2668664201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0cddd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f36e00097a0 tx=0x7f36e002b7a0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 -- 172.21.15.111:0/2668664201 shutdown_connections 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 --2- 172.21.15.111:0/2668664201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0cddd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 --2- 172.21.15.111:0/2668664201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec146a50 0x7f36ec161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 -- 172.21.15.111:0/2668664201 >> 172.21.15.111:0/2668664201 conn(0x7f36ec06a4c0 msgr2=0x7f36ec06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 -- 172.21.15.111:0/2668664201 shutdown_connections 2023-12-04T05:39:17.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.676+0000 7f36f1a94640 1 -- 172.21.15.111:0/2668664201 wait complete. 2023-12-04T05:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.677+0000 7f36f1a94640 1 Processor -- start 2023-12-04T05:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.677+0000 7f36f1a94640 1 -- start start 2023-12-04T05:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec0b2ba0 0x7f36ec141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36ec146a50 0x7f36ec0ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0abaa0 con 0x7f36ec0cd970 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec155090 con 0x7f36ec146a50 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec152270 con 0x7f36ec0b2ba0 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0ab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47476/0 (socket says 172.21.15.111:47476) 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 -- 172.21.15.111:0/4017732950 learned_addr learned my addr 172.21.15.111:0/4017732950 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eb7fe640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec0b2ba0 0x7f36ec141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36ebfff640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36ec146a50 0x7f36ec0ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 -- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec0b2ba0 msgr2=0x7f36ec141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec0b2ba0 0x7f36ec141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 -- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36ec146a50 msgr2=0x7f36ec0ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36ec146a50 0x7f36ec0ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 -- 172.21.15.111:0/4017732950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0009400 con 0x7f36ec0cd970 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eb7fe640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec0b2ba0 0x7f36ec141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36eaffd640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0ab020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f36e00098c0 tx=0x7f36e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0033c30 con 0x7f36ec0cd970 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0033d90 con 0x7f36ec0cd970 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec0b63b0 con 0x7f36ec0cd970 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e003c640 con 0x7f36ec0cd970 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.678+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec1524c0 con 0x7f36ec0cd970 2023-12-04T05:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.679+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec0693d0 con 0x7f36ec0cd970 2023-12-04T05:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.680+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f36e003a040 con 0x7f36ec0cd970 2023-12-04T05:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.680+0000 7f36d3fff640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f36d806b2a0 0x7f36d806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.680+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36e00ab640 con 0x7f36ec0cd970 2023-12-04T05:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.680+0000 7f36eb7fe640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f36d806b2a0 0x7f36d806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.681+0000 7f36eb7fe640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f36d806b2a0 0x7f36d806d760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00b820 tx=0x7f36dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.684+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36e0076660 con 0x7f36ec0cd970 2023-12-04T05:39:17.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.851+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36ec145510 con 0x7f36ec0cd970 2023-12-04T05:39:17.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.852+0000 7f36d3fff640 1 -- 172.21.15.111:0/4017732950 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f36e0079c80 con 0x7f36ec0cd970 2023-12-04T05:39:17.854 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:17.855 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":19,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:16.542580+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:17.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f36d806b2a0 msgr2=0x7f36d806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f36d806b2a0 0x7f36d806d760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00b820 tx=0x7f36dc008040 comp rx=0 tx=0).stop 2023-12-04T05:39:17.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 msgr2=0x7f36ec0ab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0ab020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f36e00098c0 tx=0x7f36e00046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 shutdown_connections 2023-12-04T05:39:17.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f36d806b2a0 0x7f36d806d760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36ec146a50 0x7f36ec0ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36ec0cd970 0x7f36ec0ab020 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 --2- 172.21.15.111:0/4017732950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36ec0b2ba0 0x7f36ec141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.854+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 >> 172.21.15.111:0/4017732950 conn(0x7f36ec06a4c0 msgr2=0x7f36ec0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:17.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.855+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 shutdown_connections 2023-12-04T05:39:17.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.855+0000 7f36f1a94640 1 -- 172.21.15.111:0/4017732950 wait complete. 2023-12-04T05:39:17.867 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-04T05:39:17.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.968+0000 7fad13505640 1 Processor -- start 2023-12-04T05:39:17.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.968+0000 7fad13505640 1 -- start start 2023-12-04T05:39:17.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad13505640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c06bc90 0x7fad0c06b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad13505640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad0c0b3590 0x7fad0c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad13505640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0cf230 con 0x7fad0c15d6c0 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad13505640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c06bb10 con 0x7fad0c0b3590 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad13505640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0cf4d0 con 0x7fad0c06bc90 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad10aa0640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fad0c15d6c0 0x7fad0c06c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32898/0 (socket says 172.21.15.111:32898) 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad10aa0640 1 -- 172.21.15.111:0/3760576187 learned_addr learned my addr 172.21.15.111:0/3760576187 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.969+0000 7fad11aa2640 1 --2- 172.21.15.111:0/3760576187 >> v2:172.21.15.111:3300/0 conn(0x7fad0c0b3590 0x7fad0c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad112a1640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c06bc90 0x7fad0c06b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad11aa2640 1 -- 172.21.15.111:0/3760576187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c06bc90 msgr2=0x7fad0c06b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad11aa2640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c06bc90 0x7fad0c06b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad11aa2640 1 -- 172.21.15.111:0/3760576187 >> v1:172.21.15.111:6789/0 conn(0x7fad0c15d6c0 legacy=0x7fad0c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad11aa2640 1 -- 172.21.15.111:0/3760576187 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0cf7b0 con 0x7fad0c0b3590 2023-12-04T05:39:17.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad03fff640 1 -- 172.21.15.111:0/3760576187 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249920788 0 0) 0x7fad0c0cf230 con 0x7fad0c15d6c0 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad112a1640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c06bc90 0x7fad0c06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.970+0000 7fad11aa2640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c0b3590 0x7fad0c0cb5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad040097a0 tx=0x7fad0402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1b24e479bc64aa server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad03fff640 1 -- 172.21.15.111:0/3760576187 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0402e070 con 0x7fad0c0b3590 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad03fff640 1 -- 172.21.15.111:0/3760576187 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad040315a0 con 0x7fad0c0b3590 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad03fff640 1 -- 172.21.15.111:0/3760576187 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04031930 con 0x7fad0c0b3590 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 -- 172.21.15.111:0/3760576187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c0b3590 msgr2=0x7fad0c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c0b3590 0x7fad0c0cb5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad040097a0 tx=0x7fad0402b840 comp rx=0 tx=0).stop 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 -- 172.21.15.111:0/3760576187 shutdown_connections 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c0b3590 0x7fad0c0cb5b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 --2- 172.21.15.111:0/3760576187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c06bc90 0x7fad0c06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 -- 172.21.15.111:0/3760576187 >> 172.21.15.111:0/3760576187 conn(0x7fad0c0b2a70 msgr2=0x7fad0c0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 -- 172.21.15.111:0/3760576187 shutdown_connections 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.971+0000 7fad13505640 1 -- 172.21.15.111:0/3760576187 wait complete. 2023-12-04T05:39:17.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.972+0000 7fad13505640 1 Processor -- start 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.972+0000 7fad13505640 1 -- start start 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad13505640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 0x7fad0c1f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad13505640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad0c0b3590 0x7fad0c160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad13505640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c0b6970 0x7fad0c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad13505640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0afe40 con 0x7fad0c06bc90 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad13505640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b0260 con 0x7fad0c0b3590 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad13505640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c1f2b90 con 0x7fad0c0b6970 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 0x7fad0c1f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 0x7fad0c1f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47480/0 (socket says 172.21.15.111:47480) 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 -- 172.21.15.111:0/1849093600 learned_addr learned my addr 172.21.15.111:0/1849093600 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 -- 172.21.15.111:0/1849093600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c0b6970 msgr2=0x7fad0c0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c0b6970 0x7fad0c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 -- 172.21.15.111:0/1849093600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad0c0b3590 msgr2=0x7fad0c160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad0c0b3590 0x7fad0c160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.973+0000 7fad112a1640 1 -- 172.21.15.111:0/1849093600 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04009400 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.974+0000 7fad112a1640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 0x7fad0c1f2650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad08007f20 tx=0x7fad0800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.974+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800dee0 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.974+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800f040 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.974+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c15db50 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.974+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08003db0 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.974+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c15e110 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.975+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c067460 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.975+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fad08004030 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.976+0000 7fad01ffb640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7facf006b2a0 0x7facf006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.976+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad08089da0 con 0x7fad0c06bc90 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.976+0000 7fad10aa0640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7facf006b2a0 0x7facf006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:17.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.976+0000 7fad10aa0640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7facf006b2a0 0x7facf006d760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:17.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:17.980+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad08054cc0 con 0x7fad0c06bc90 2023-12-04T05:39:18.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.186+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fad0c0b6ee0 con 0x7fad0c06bc90 2023-12-04T05:39:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.223+0000 7fad01ffb640 1 -- 172.21.15.111:0/1849093600 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fad0801a0b0 con 0x7fad0c06bc90 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7facf006b2a0 msgr2=0x7facf006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7facf006b2a0 0x7facf006d760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc007960 comp rx=0 tx=0).stop 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 msgr2=0x7fad0c1f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 0x7fad0c1f2650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad08007f20 tx=0x7fad0800d560 comp rx=0 tx=0).stop 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 shutdown_connections 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7facf006b2a0 0x7facf006d760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad0c0b6970 0x7fad0c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad0c0b3590 0x7fad0c160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 --2- 172.21.15.111:0/1849093600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad0c06bc90 0x7fad0c1f2650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 >> 172.21.15.111:0/1849093600 conn(0x7fad0c0b2a70 msgr2=0x7fad0c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:18.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 shutdown_connections 2023-12-04T05:39:18.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.225+0000 7fad13505640 1 -- 172.21.15.111:0/1849093600 wait complete. 2023-12-04T05:39:18.237 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-04T05:39:18.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-04T05:39:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.338+0000 7f6cd7489640 1 Processor -- start 2023-12-04T05:39:18.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.338+0000 7f6cd7489640 1 -- start start 2023-12-04T05:39:18.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd7489640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0150680 0x7f6cd0071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd7489640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6cd006cac0 0x7f6cd006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd7489640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cd0159ce0 0x7f6cd015a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd7489640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd0072390 con 0x7f6cd0159ce0 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd7489640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd015acb0 con 0x7f6cd006cac0 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd7489640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd006d480 con 0x7f6cd0150680 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cd0159ce0 0x7f6cd015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cd0159ce0 0x7f6cd015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47490/0 (socket says 172.21.15.111:47490) 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 -- 172.21.15.111:0/1712407360 learned_addr learned my addr 172.21.15.111:0/1712407360 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5225640 1 --2- 172.21.15.111:0/1712407360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0150680 0x7f6cd0071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd4a24640 1 --2- 172.21.15.111:0/1712407360 >> v2:172.21.15.192:3300/0 conn(0x7f6cd006cac0 0x7f6cd006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 -- 172.21.15.111:0/1712407360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0150680 msgr2=0x7f6cd0071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 --2- 172.21.15.111:0/1712407360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0150680 0x7f6cd0071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 -- 172.21.15.111:0/1712407360 >> v2:172.21.15.192:3300/0 conn(0x7f6cd006cac0 msgr2=0x7f6cd006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 --2- 172.21.15.111:0/1712407360 >> v2:172.21.15.192:3300/0 conn(0x7f6cd006cac0 0x7f6cd006cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.339+0000 7f6cd5a26640 1 -- 172.21.15.111:0/1712407360 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd015a9d0 con 0x7f6cd0159ce0 2023-12-04T05:39:18.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.340+0000 7f6cd5a26640 1 --2- 172.21.15.111:0/1712407360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0159ce0 0x7f6cd015a150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0067ee0 tx=0x7f6ccc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542c4b207f40b5d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:18.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.340+0000 7f6cc7fff640 1 -- 172.21.15.111:0/1712407360 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc031870 con 0x7f6cd0159ce0 2023-12-04T05:39:18.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.340+0000 7f6cc7fff640 1 -- 172.21.15.111:0/1712407360 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0319d0 con 0x7f6cd0159ce0 2023-12-04T05:39:18.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.340+0000 7f6cc7fff640 1 -- 172.21.15.111:0/1712407360 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc031d60 con 0x7f6cd0159ce0 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 -- 172.21.15.111:0/1712407360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0159ce0 msgr2=0x7f6cd015a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 --2- 172.21.15.111:0/1712407360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0159ce0 0x7f6cd015a150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0067ee0 tx=0x7f6ccc02b910 comp rx=0 tx=0).stop 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 -- 172.21.15.111:0/1712407360 shutdown_connections 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 --2- 172.21.15.111:0/1712407360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0159ce0 0x7f6cd015a150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 --2- 172.21.15.111:0/1712407360 >> v2:172.21.15.192:3300/0 conn(0x7f6cd006cac0 0x7f6cd006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 --2- 172.21.15.111:0/1712407360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0150680 0x7f6cd0071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 -- 172.21.15.111:0/1712407360 >> 172.21.15.111:0/1712407360 conn(0x7f6cd006a4c0 msgr2=0x7f6cd006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 -- 172.21.15.111:0/1712407360 shutdown_connections 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.341+0000 7f6cd7489640 1 -- 172.21.15.111:0/1712407360 wait complete. 2023-12-04T05:39:18.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.342+0000 7f6cd7489640 1 Processor -- start 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.342+0000 7f6cd7489640 1 -- start start 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd006cac0 0x7f6cd014a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 0x7f6cd00afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0159ce0 0x7f6cd0146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd014b440 con 0x7f6cd0150680 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd014ff50 con 0x7f6cd006cac0 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b0a50 con 0x7f6cd0159ce0 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 0x7f6cd00afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 0x7f6cd00afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47500/0 (socket says 172.21.15.111:47500) 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 -- 172.21.15.111:0/3013932621 learned_addr learned my addr 172.21.15.111:0/3013932621 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd5225640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd006cac0 0x7f6cd014a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd5a26640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0159ce0 0x7f6cd0146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 -- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0159ce0 msgr2=0x7f6cd0146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0159ce0 0x7f6cd0146c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 -- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd006cac0 msgr2=0x7f6cd014a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd006cac0 0x7f6cd014a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 -- 172.21.15.111:0/3013932621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0094d0 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd4a24640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 0x7f6cd00afa40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000d800 tx=0x7f6cc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000b6c0 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc000b820 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd014b160 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.343+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000bbb0 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.344+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd014f750 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.345+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd00693d0 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.345+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6cc000bd10 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.345+0000 7f6cc5ffb640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6cb406b2a0 0x7f6cb406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.345+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cc008a0a0 con 0x7f6cd0150680 2023-12-04T05:39:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.345+0000 7f6cd5225640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6cb406b2a0 0x7f6cb406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.345+0000 7f6cd5225640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6cb406b2a0 0x7f6cb406d760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003aa0 tx=0x7f6cc80098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.349+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cc0054f10 con 0x7f6cd0150680 2023-12-04T05:39:18.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:18.554+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6cd00cecb0 con 0x7f6cd0150680 2023-12-04T05:39:19.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.227+0000 7f6cc5ffb640 1 -- 172.21.15.111:0/3013932621 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6cc0058530 con 0x7f6cd0150680 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6cb406b2a0 msgr2=0x7f6cb406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6cb406b2a0 0x7f6cb406d760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003aa0 tx=0x7f6cc80098d0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 msgr2=0x7f6cd00afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 0x7f6cd00afa40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000d800 tx=0x7f6cc000dcc0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 shutdown_connections 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6cb406b2a0 0x7f6cb406d760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cd0159ce0 0x7f6cd0146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cd0150680 0x7f6cd00afa40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 --2- 172.21.15.111:0/3013932621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd006cac0 0x7f6cd014a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 >> 172.21.15.111:0/3013932621 conn(0x7f6cd006a4c0 msgr2=0x7f6cd00b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 shutdown_connections 2023-12-04T05:39:19.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.230+0000 7f6cd7489640 1 -- 172.21.15.111:0/3013932621 wait complete. 2023-12-04T05:39:19.244 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-04T05:39:19.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:19.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.345+0000 7faa6a8e7640 1 Processor -- start 2023-12-04T05:39:19.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.345+0000 7faa6a8e7640 1 -- start start 2023-12-04T05:39:19.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa6a8e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa6406ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa6a8e7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa6415ed10 0x7faa6415f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa6a8e7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faa6406fd20 0x7faa64070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa6a8e7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6406f100 con 0x7faa6415ed10 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa6a8e7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6413e380 con 0x7faa6406fd20 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa6a8e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6415b810 con 0x7faa6406e7e0 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa637fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa6415ed10 0x7faa6415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa637fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa6415ed10 0x7faa6415f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47508/0 (socket says 172.21.15.111:47508) 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa637fe640 1 -- 172.21.15.111:0/242147647 learned_addr learned my addr 172.21.15.111:0/242147647 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa63fff640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa6406ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.346+0000 7faa68e84640 1 --2- 172.21.15.111:0/242147647 >> v2:172.21.15.192:3300/0 conn(0x7faa6406fd20 0x7faa64070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa637fe640 1 -- 172.21.15.111:0/242147647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 msgr2=0x7faa6406ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa637fe640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa6406ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa637fe640 1 -- 172.21.15.111:0/242147647 >> v2:172.21.15.192:3300/0 conn(0x7faa6406fd20 msgr2=0x7faa64070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:19.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa637fe640 1 --2- 172.21.15.111:0/242147647 >> v2:172.21.15.192:3300/0 conn(0x7faa6406fd20 0x7faa64070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa637fe640 1 -- 172.21.15.111:0/242147647 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6415baf0 con 0x7faa6415ed10 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa63fff640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa6406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa637fe640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6415ed10 0x7faa6415f520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa640ceac0 tx=0x7faa5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526ff5da9f2d3bd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa62ffd640 1 -- 172.21.15.111:0/242147647 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580317e0 con 0x7faa6415ed10 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa62ffd640 1 -- 172.21.15.111:0/242147647 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58031940 con 0x7faa6415ed10 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.347+0000 7faa62ffd640 1 -- 172.21.15.111:0/242147647 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58031cd0 con 0x7faa6415ed10 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 -- 172.21.15.111:0/242147647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6415ed10 msgr2=0x7faa6415f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6415ed10 0x7faa6415f520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa640ceac0 tx=0x7faa5802b880 comp rx=0 tx=0).stop 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 -- 172.21.15.111:0/242147647 shutdown_connections 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/242147647 >> v2:172.21.15.192:3300/0 conn(0x7faa6406fd20 0x7faa64070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6415ed10 0x7faa6415f520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/242147647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa6406ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 -- 172.21.15.111:0/242147647 >> 172.21.15.111:0/242147647 conn(0x7faa6406a8b0 msgr2=0x7faa6406acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:19.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 -- 172.21.15.111:0/242147647 shutdown_connections 2023-12-04T05:39:19.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.348+0000 7faa6a8e7640 1 -- 172.21.15.111:0/242147647 wait complete. 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.349+0000 7faa6a8e7640 1 Processor -- start 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.349+0000 7faa6a8e7640 1 -- start start 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa6a8e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa64155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa6a8e7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 0x7faa64139e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa6a8e7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa6415ed10 0x7faa641383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa6a8e7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa64139a50 con 0x7faa6406fd20 2023-12-04T05:39:19.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa6a8e7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa64138dd0 con 0x7faa6415ed10 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa6a8e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa641390b0 con 0x7faa6406e7e0 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 0x7faa64139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 0x7faa64139e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47516/0 (socket says 172.21.15.111:47516) 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 -- 172.21.15.111:0/2736218333 learned_addr learned my addr 172.21.15.111:0/2736218333 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa63fff640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa64155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa68e84640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa6415ed10 0x7faa641383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 -- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 msgr2=0x7faa64155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa64155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 -- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa6415ed10 msgr2=0x7faa641383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa6415ed10 0x7faa641383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa637fe640 1 -- 172.21.15.111:0/2736218333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580094d0 con 0x7faa6406fd20 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.350+0000 7faa63fff640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa64155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.351+0000 7faa637fe640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 0x7faa64139e70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c00 tx=0x7faa580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.351+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58032880 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.351+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58032a20 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.351+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa64139390 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.351+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58032dd0 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.351+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa641557c0 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.352+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa64139b90 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.352+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7faa5804e450 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.352+0000 7faa60ff9640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa5006b2d0 0x7faa5006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.353+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa580abc40 con 0x7faa6406fd20 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.353+0000 7faa63fff640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa5006b2d0 0x7faa5006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.353+0000 7faa63fff640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa5006b2d0 0x7faa5006d790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa640cf320 tx=0x7faa54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.357+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa58076c00 con 0x7faa6406fd20 2023-12-04T05:39:19.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.558+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa640693d0 con 0x7faa6406fd20 2023-12-04T05:39:19.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.558+0000 7faa60ff9640 1 -- 172.21.15.111:0/2736218333 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7faa5807a220 con 0x7faa6406fd20 2023-12-04T05:39:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:19.560 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:19.560 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:19.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa5006b2d0 msgr2=0x7faa5006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa5006b2d0 0x7faa5006d790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa640cf320 tx=0x7faa54008040 comp rx=0 tx=0).stop 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 msgr2=0x7faa64139e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 0x7faa64139e70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c00 tx=0x7faa580046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 shutdown_connections 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa5006b2d0 0x7faa5006d790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa6415ed10 0x7faa641383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6406fd20 0x7faa64139e70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 --2- 172.21.15.111:0/2736218333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa6406e7e0 0x7faa64155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 >> 172.21.15.111:0/2736218333 conn(0x7faa6406a8b0 msgr2=0x7faa640b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 shutdown_connections 2023-12-04T05:39:19.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.560+0000 7faa6a8e7640 1 -- 172.21.15.111:0/2736218333 wait complete. 2023-12-04T05:39:19.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.670+0000 7f05df11d640 1 Processor -- start 2023-12-04T05:39:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.670+0000 7f05df11d640 1 -- start start 2023-12-04T05:39:19.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05df11d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f05d806b4d0 0x7f05d806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05df11d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05d80add00 0x7f05d80ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05df11d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d806be70 con 0x7f05d81544c0 2023-12-04T05:39:19.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05df11d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d80ad5d0 con 0x7f05d80add00 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05df11d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d806df90 con 0x7f05d806b4d0 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dd6ba640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05d80add00 0x7f05d80ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dceb9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f05d81544c0 0x7f05d806cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32904/0 (socket says 172.21.15.111:32904) 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dceb9640 1 -- 172.21.15.111:0/3839012806 learned_addr learned my addr 172.21.15.111:0/3839012806 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05d7fff640 1 --2- 172.21.15.111:0/3839012806 >> v2:172.21.15.192:3300/0 conn(0x7f05d806b4d0 0x7f05d806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dd6ba640 1 -- 172.21.15.111:0/3839012806 >> v2:172.21.15.192:3300/0 conn(0x7f05d806b4d0 msgr2=0x7f05d806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dd6ba640 1 --2- 172.21.15.111:0/3839012806 >> v2:172.21.15.192:3300/0 conn(0x7f05d806b4d0 0x7f05d806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dd6ba640 1 -- 172.21.15.111:0/3839012806 >> v1:172.21.15.111:6789/0 conn(0x7f05d81544c0 legacy=0x7f05d806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05d77fe640 1 -- 172.21.15.111:0/3839012806 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313627414 0 0) 0x7f05d806be70 con 0x7f05d81544c0 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.671+0000 7f05dd6ba640 1 -- 172.21.15.111:0/3839012806 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d81548a0 con 0x7f05d80add00 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.672+0000 7f05dd6ba640 1 --2- 172.21.15.111:0/3839012806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d80add00 0x7f05d80ae190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f05d8066460 tx=0x7f05d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5ff83ba3c84c68 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.672+0000 7f05d77fe640 1 -- 172.21.15.111:0/3839012806 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0031830 con 0x7f05d80add00 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.672+0000 7f05d77fe640 1 -- 172.21.15.111:0/3839012806 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d0031990 con 0x7f05d80add00 2023-12-04T05:39:19.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.672+0000 7f05d77fe640 1 -- 172.21.15.111:0/3839012806 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0031d20 con 0x7f05d80add00 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 -- 172.21.15.111:0/3839012806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d80add00 msgr2=0x7f05d80ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 --2- 172.21.15.111:0/3839012806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d80add00 0x7f05d80ae190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f05d8066460 tx=0x7f05d002b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 -- 172.21.15.111:0/3839012806 shutdown_connections 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 --2- 172.21.15.111:0/3839012806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d80add00 0x7f05d80ae190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 --2- 172.21.15.111:0/3839012806 >> v2:172.21.15.192:3300/0 conn(0x7f05d806b4d0 0x7f05d806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 -- 172.21.15.111:0/3839012806 >> 172.21.15.111:0/3839012806 conn(0x7f05d80b1b50 msgr2=0x7f05d8070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 -- 172.21.15.111:0/3839012806 shutdown_connections 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.673+0000 7f05df11d640 1 -- 172.21.15.111:0/3839012806 wait complete. 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.674+0000 7f05df11d640 1 Processor -- start 2023-12-04T05:39:19.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.674+0000 7f05df11d640 1 -- start start 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05df11d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 0x7f05d80af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05df11d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05d80add00 0x7f05d814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05df11d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05d81544c0 0x7f05d814baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05df11d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d80b2d80 con 0x7f05d806b4d0 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05df11d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d814e4c0 con 0x7f05d81544c0 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05df11d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d814e7a0 con 0x7f05d80add00 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 0x7f05d80af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 0x7f05d80af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47530/0 (socket says 172.21.15.111:47530) 2023-12-04T05:39:19.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 -- 172.21.15.111:0/3601926960 learned_addr learned my addr 172.21.15.111:0/3601926960 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 -- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05d80add00 msgr2=0x7f05d814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05d80add00 0x7f05d814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 -- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05d81544c0 msgr2=0x7f05d814baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dd6ba640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05d81544c0 0x7f05d814baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05d81544c0 0x7f05d814baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 -- 172.21.15.111:0/3601926960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d0009400 con 0x7f05d806b4d0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05dceb9640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 0x7f05d80af600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c430 tx=0x7f05c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.675+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c00074e0 con 0x7f05d806b4d0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.676+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c0007660 con 0x7f05d806b4d0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.676+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0004a20 con 0x7f05d806b4d0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.676+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d80b2a10 con 0x7f05d806b4d0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.676+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d80afb40 con 0x7f05d806b4d0 2023-12-04T05:39:19.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.676+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d80693d0 con 0x7f05d806b4d0 2023-12-04T05:39:19.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.677+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f05c0004b80 con 0x7f05d806b4d0 2023-12-04T05:39:19.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.677+0000 7f05d57fa640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05b806b2a0 0x7f05b806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.678+0000 7f05d7fff640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05b806b2a0 0x7f05b806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.678+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05c0089a60 con 0x7f05d806b4d0 2023-12-04T05:39:19.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.678+0000 7f05d7fff640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05b806b2a0 0x7f05b806d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05c80070b0 tx=0x7f05c8007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.682+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05c00548d0 con 0x7f05d806b4d0 2023-12-04T05:39:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.848+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05d8071bf0 con 0x7f05d806b4d0 2023-12-04T05:39:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.849+0000 7f05d57fa640 1 -- 172.21.15.111:0/3601926960 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f05c0057ef0 con 0x7f05d806b4d0 2023-12-04T05:39:19.851 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:19.851 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05b806b2a0 msgr2=0x7f05b806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05b806b2a0 0x7f05b806d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05c80070b0 tx=0x7f05c8007040 comp rx=0 tx=0).stop 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 msgr2=0x7f05d80af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 0x7f05d80af600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c430 tx=0x7f05c000c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 shutdown_connections 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05b806b2a0 0x7f05b806d760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05d81544c0 0x7f05d814baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05d80add00 0x7f05d814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 --2- 172.21.15.111:0/3601926960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d806b4d0 0x7f05d80af600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 >> 172.21.15.111:0/3601926960 conn(0x7f05d80b1b50 msgr2=0x7f05d80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 shutdown_connections 2023-12-04T05:39:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.851+0000 7f05df11d640 1 -- 172.21.15.111:0/3601926960 wait complete. 2023-12-04T05:39:19.864 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-04T05:39:19.864 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:39:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 Processor -- start 2023-12-04T05:39:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 -- start start 2023-12-04T05:39:19.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0d1780 0x7fd15c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd15c0b2ad0 0x7fd15c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c0cdcb0 con 0x7fd15c0b20a0 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c0d39e0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd1619ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c0d3cc0 con 0x7fd15c0d1780 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.962+0000 7fd15b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd15c0b2ad0 0x7fd15c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd15c0b2ad0 0x7fd15c0ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47532/0 (socket says 172.21.15.111:47532) 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 -- 172.21.15.111:0/3074632292 learned_addr learned my addr 172.21.15.111:0/3074632292 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15affd640 1 --2- 172.21.15.111:0/3074632292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0d1780 0x7fd15c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 -- 172.21.15.111:0/3074632292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0d1780 msgr2=0x7fd15c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 --2- 172.21.15.111:0/3074632292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0d1780 0x7fd15c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 -- 172.21.15.111:0/3074632292 >> v1:172.21.15.192:6789/0 conn(0x7fd15c0b20a0 legacy=0x7fd15c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 -- 172.21.15.111:0/3074632292 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c0cdf50 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.963+0000 7fd15b7fe640 1 --2- 172.21.15.111:0/3074632292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c0ce750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd154009e50 tx=0x7fd15402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc1f935909ec29 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd159ffb640 1 -- 172.21.15.111:0/3074632292 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15402ca70 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd159ffb640 1 -- 172.21.15.111:0/3074632292 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15402cbd0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd159ffb640 1 -- 172.21.15.111:0/3074632292 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1540343f0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 -- 172.21.15.111:0/3074632292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 msgr2=0x7fd15c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 --2- 172.21.15.111:0/3074632292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c0ce750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd154009e50 tx=0x7fd15402bed0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 -- 172.21.15.111:0/3074632292 shutdown_connections 2023-12-04T05:39:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 --2- 172.21.15.111:0/3074632292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c0ce750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 --2- 172.21.15.111:0/3074632292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0d1780 0x7fd15c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 -- 172.21.15.111:0/3074632292 >> 172.21.15.111:0/3074632292 conn(0x7fd15c09c4e0 msgr2=0x7fd15c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:19.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 -- 172.21.15.111:0/3074632292 shutdown_connections 2023-12-04T05:39:19.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.964+0000 7fd1619ce640 1 -- 172.21.15.111:0/3074632292 wait complete. 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.965+0000 7fd1619ce640 1 Processor -- start 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.965+0000 7fd1619ce640 1 -- start start 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd1619ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0b20a0 0x7fd15c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd1619ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd1619ce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd15c0d1780 0x7fd15c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd1619ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c15a1a0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd1619ce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c160d00 con 0x7fd15c0d1780 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd1619ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c159be0 con 0x7fd15c0b20a0 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47536/0 (socket says 172.21.15.111:47536) 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 -- 172.21.15.111:0/250617437 learned_addr learned my addr 172.21.15.111:0/250617437 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 -- 172.21.15.111:0/250617437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0b20a0 msgr2=0x7fd15c0b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0b20a0 0x7fd15c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 -- 172.21.15.111:0/250617437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd15c0d1780 msgr2=0x7fd15c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd15c0d1780 0x7fd15c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.966+0000 7fd15a7fc640 1 -- 172.21.15.111:0/250617437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154009b30 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.967+0000 7fd15a7fc640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c156c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c450 tx=0x7fd15000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.967+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500052d0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.967+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150005430 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.967+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c159ec0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.967+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150012870 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.967+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c157170 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.968+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15c15ddd0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.968+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd1500129d0 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.969+0000 7fd143fff640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd14806b2a0 0x7fd14806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:19.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.969+0000 7fd15affd640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd14806b2a0 0x7fd14806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:19.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.969+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd150089710 con 0x7fd15c0b2ad0 2023-12-04T05:39:19.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.969+0000 7fd15affd640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd14806b2a0 0x7fd14806d760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:19.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:19.973+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd15c15ddd0 con 0x7fd15c0b2ad0 2023-12-04T05:39:20.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.137+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd15c15d900 con 0x7fd14806b2a0 2023-12-04T05:39:20.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.143+0000 7fd143fff640 1 -- 172.21.15.111:0/250617437 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249835 (secure 0 0 0) 0x7fd14c00be20 con 0x7fd14806b2a0 2023-12-04T05:39:20.147 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:20.149 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:39:20.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd14806b2a0 msgr2=0x7fd14806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd14806b2a0 0x7fd14806d760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c008040 comp rx=0 tx=0).stop 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 msgr2=0x7fd15c156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c156c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c450 tx=0x7fd15000c910 comp rx=0 tx=0).stop 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 shutdown_connections 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd14806b2a0 0x7fd14806d760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd15c0d1780 0x7fd15c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd15c0b2ad0 0x7fd15c156c30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 --2- 172.21.15.111:0/250617437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd15c0b20a0 0x7fd15c0b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.146+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 >> 172.21.15.111:0/250617437 conn(0x7fd15c09c4e0 msgr2=0x7fd15c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.147+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 shutdown_connections 2023-12-04T05:39:20.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.147+0000 7fd1619ce640 1 -- 172.21.15.111:0/250617437 wait complete. 2023-12-04T05:39:20.158 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-12-04T05:39:18.472939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218556,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756164,"statfs":{"total":773094113280,"available":772870311936,"internally_reserved":0,"allocated":2433024,"data_stored":1546680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000624"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-12-04T05:39:08.012312+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:39:08.012312+0000","last_peered":"2023-12-04T05:39:08.012312+0000","last_clean":"2023-12-04T05:39:08.012312+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:39:08.012312+0000","last_undegraded":"2023-12-04T05:39:08.012312+0000","last_fullsized":"2023-12-04T05:39:08.012312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:16.197699+0000","last_change":"2023-12-04T05:39:16.197699+0000","last_active":"2023-12-04T05:39:16.197699+0000","last_peered":"2023-12-04T05:39:16.197699+0000","last_clean":"2023-12-04T05:39:16.197699+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:16.197699+0000","last_undegraded":"2023-12-04T05:39:16.197699+0000","last_fullsized":"2023-12-04T05:39:16.197699+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:39:15.192640+0000","last_change":"2023-12-04T05:39:15.192640+0000","last_active":"2023-12-04T05:39:15.192640+0000","last_peered":"2023-12-04T05:39:15.192640+0000","last_clean":"2023-12-04T05:39:15.192640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:39:15.192640+0000","last_undegraded":"2023-12-04T05:39:15.192640+0000","last_fullsized":"2023-12-04T05:39:15.192640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":636111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:39:20.163 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:39:20.204 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-12-04T05:39:20.203+0000 7f3082083640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-04T05:39:20.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.299+0000 7f28c883d640 1 Processor -- start 2023-12-04T05:39:20.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.299+0000 7f28c883d640 1 -- start start 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c883d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28c40b4a00 0x7f28c40d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c883d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c40d15f0 0x7f28c406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c883d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c4161d10 con 0x7f28c41460c0 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c883d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c4146fa0 con 0x7f28c40b4a00 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c883d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c415d990 con 0x7f28c40d15f0 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c1d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28c40b4a00 0x7f28c40d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c1d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28c40b4a00 0x7f28c40d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47540/0 (socket says 172.21.15.111:47540) 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c1d9b640 1 -- 172.21.15.111:0/287704079 learned_addr learned my addr 172.21.15.111:0/287704079 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:20.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c2d9d640 1 --2- 172.21.15.111:0/287704079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c40d15f0 0x7f28c406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c1d9b640 1 -- 172.21.15.111:0/287704079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c40d15f0 msgr2=0x7f28c406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:20.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c1d9b640 1 --2- 172.21.15.111:0/287704079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c40d15f0 0x7f28c406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.300+0000 7f28c1d9b640 1 -- 172.21.15.111:0/287704079 >> v1:172.21.15.192:6789/0 conn(0x7f28c41460c0 legacy=0x7f28c41464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.301+0000 7f28c1d9b640 1 -- 172.21.15.111:0/287704079 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40b4e80 con 0x7f28c40b4a00 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.301+0000 7f28c1d9b640 1 --2- 172.21.15.111:0/287704079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40b4a00 0x7f28c40d10b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009870 tx=0x7f28b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbac79af822517d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.301+0000 7f28c159a640 1 -- 172.21.15.111:0/287704079 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8030840 con 0x7f28c40b4a00 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.301+0000 7f28c159a640 1 -- 172.21.15.111:0/287704079 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b80309a0 con 0x7f28c40b4a00 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.301+0000 7f28c159a640 1 -- 172.21.15.111:0/287704079 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8030d30 con 0x7f28c40b4a00 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 -- 172.21.15.111:0/287704079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40b4a00 msgr2=0x7f28c40d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 --2- 172.21.15.111:0/287704079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40b4a00 0x7f28c40d10b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009870 tx=0x7f28b802b980 comp rx=0 tx=0).stop 2023-12-04T05:39:20.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 -- 172.21.15.111:0/287704079 shutdown_connections 2023-12-04T05:39:20.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 --2- 172.21.15.111:0/287704079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c40d15f0 0x7f28c406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 --2- 172.21.15.111:0/287704079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40b4a00 0x7f28c40d10b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 -- 172.21.15.111:0/287704079 >> 172.21.15.111:0/287704079 conn(0x7f28c406a4c0 msgr2=0x7f28c406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:20.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 -- 172.21.15.111:0/287704079 shutdown_connections 2023-12-04T05:39:20.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.302+0000 7f28c883d640 1 -- 172.21.15.111:0/287704079 wait complete. 2023-12-04T05:39:20.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.303+0000 7f28c883d640 1 Processor -- start 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.303+0000 7f28c883d640 1 -- start start 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c883d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28c40b4a00 0x7f28c4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c883d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 0x7f28c4141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c883d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c41460c0 0x7f28c41f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c883d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c415f540 con 0x7f28c40d15f0 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c883d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41604e0 con 0x7f28c40b4a00 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c883d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41422b0 con 0x7f28c41460c0 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 0x7f28c4141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 0x7f28c4141d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47544/0 (socket says 172.21.15.111:47544) 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 -- 172.21.15.111:0/1045290042 learned_addr learned my addr 172.21.15.111:0/1045290042 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:20.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c259c640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28c40b4a00 0x7f28c4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c2d9d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c41460c0 0x7f28c41f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 -- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c41460c0 msgr2=0x7f28c41f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c41460c0 0x7f28c41f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 -- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28c40b4a00 msgr2=0x7f28c4152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28c40b4a00 0x7f28c4152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c1d9b640 1 -- 172.21.15.111:0/1045290042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80094d0 con 0x7f28c40d15f0 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.304+0000 7f28c259c640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28c40b4a00 0x7f28c4152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.305+0000 7f28c1d9b640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 0x7f28c4141d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f28b802b9c0 tx=0x7f28b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.305+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8039c90 con 0x7f28c40d15f0 2023-12-04T05:39:20.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.305+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8039df0 con 0x7f28c40d15f0 2023-12-04T05:39:20.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.305+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80446c0 con 0x7f28c40d15f0 2023-12-04T05:39:20.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.305+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c4160050 con 0x7f28c40d15f0 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.305+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c4152b50 con 0x7f28c40d15f0 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.306+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c41601e0 con 0x7f28c40d15f0 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.306+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f28b802f390 con 0x7f28c40d15f0 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.306+0000 7f28b2ffd640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f28a406b2a0 0x7f28a406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.307+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28b803e070 con 0x7f28c40d15f0 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.307+0000 7f28c259c640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f28a406b2a0 0x7f28a406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.307+0000 7f28c259c640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f28a406b2a0 0x7f28a406d760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28b40012e0 tx=0x7f28b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.311+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28b8077410 con 0x7f28c40d15f0 2023-12-04T05:39:20.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.476+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28c41616c0 con 0x7f28a406b2a0 2023-12-04T05:39:20.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.482+0000 7f28b2ffd640 1 -- 172.21.15.111:0/1045290042 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264576 (secure 0 0 0) 0x7f28b4001ad0 con 0x7f28a406b2a0 2023-12-04T05:39:20.486 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:20.488 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:39:20.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f28a406b2a0 msgr2=0x7f28a406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f28a406b2a0 0x7f28a406d760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28b40012e0 tx=0x7f28b4009040 comp rx=0 tx=0).stop 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 msgr2=0x7f28c4141d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 0x7f28c4141d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f28b802b9c0 tx=0x7f28b8004730 comp rx=0 tx=0).stop 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 shutdown_connections 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f28a406b2a0 0x7f28a406d760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28c41460c0 0x7f28c41f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28c40d15f0 0x7f28c4141d70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 --2- 172.21.15.111:0/1045290042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28c40b4a00 0x7f28c4152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 >> 172.21.15.111:0/1045290042 conn(0x7f28c406a4c0 msgr2=0x7f28c40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 shutdown_connections 2023-12-04T05:39:20.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.485+0000 7f28c883d640 1 -- 172.21.15.111:0/1045290042 wait complete. 2023-12-04T05:39:20.497 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-12-04T05:39:20.473254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.275202"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519836+0000","last_change":"2023-12-04T05:39:17.207470+0000","last_active":"2023-12-04T05:39:18.519836+0000","last_peered":"2023-12-04T05:39:18.519836+0000","last_clean":"2023-12-04T05:39:18.519836+0000","last_became_active":"2023-12-04T05:39:17.207195+0000","last_became_peered":"2023-12-04T05:39:17.207195+0000","last_unstale":"2023-12-04T05:39:18.519836+0000","last_undegraded":"2023-12-04T05:39:18.519836+0000","last_fullsized":"2023-12-04T05:39:18.519836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:01:28.140633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195840+0000","last_change":"2023-12-04T05:39:17.210099+0000","last_active":"2023-12-04T05:39:18.195840+0000","last_peered":"2023-12-04T05:39:18.195840+0000","last_clean":"2023-12-04T05:39:18.195840+0000","last_became_active":"2023-12-04T05:39:17.209800+0000","last_became_peered":"2023-12-04T05:39:17.209800+0000","last_unstale":"2023-12-04T05:39:18.195840+0000","last_undegraded":"2023-12-04T05:39:18.195840+0000","last_fullsized":"2023-12-04T05:39:18.195840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518434+0000","last_change":"2023-12-04T05:39:17.213084+0000","last_active":"2023-12-04T05:39:18.518434+0000","last_peered":"2023-12-04T05:39:18.518434+0000","last_clean":"2023-12-04T05:39:18.518434+0000","last_became_active":"2023-12-04T05:39:17.212742+0000","last_became_peered":"2023-12-04T05:39:17.212742+0000","last_unstale":"2023-12-04T05:39:18.518434+0000","last_undegraded":"2023-12-04T05:39:18.518434+0000","last_fullsized":"2023-12-04T05:39:18.518434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195184+0000","last_change":"2023-12-04T05:39:17.209715+0000","last_active":"2023-12-04T05:39:18.195184+0000","last_peered":"2023-12-04T05:39:18.195184+0000","last_clean":"2023-12-04T05:39:18.195184+0000","last_became_active":"2023-12-04T05:39:17.209399+0000","last_became_peered":"2023-12-04T05:39:17.209399+0000","last_unstale":"2023-12-04T05:39:18.195184+0000","last_undegraded":"2023-12-04T05:39:18.195184+0000","last_fullsized":"2023-12-04T05:39:18.195184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519183+0000","last_change":"2023-12-04T05:39:17.209067+0000","last_active":"2023-12-04T05:39:18.519183+0000","last_peered":"2023-12-04T05:39:18.519183+0000","last_clean":"2023-12-04T05:39:18.519183+0000","last_became_active":"2023-12-04T05:39:17.208794+0000","last_became_peered":"2023-12-04T05:39:17.208794+0000","last_unstale":"2023-12-04T05:39:18.519183+0000","last_undegraded":"2023-12-04T05:39:18.519183+0000","last_fullsized":"2023-12-04T05:39:18.519183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195035+0000","last_change":"2023-12-04T05:39:17.216846+0000","last_active":"2023-12-04T05:39:18.195035+0000","last_peered":"2023-12-04T05:39:18.195035+0000","last_clean":"2023-12-04T05:39:18.195035+0000","last_became_active":"2023-12-04T05:39:17.216536+0000","last_became_peered":"2023-12-04T05:39:17.216536+0000","last_unstale":"2023-12-04T05:39:18.195035+0000","last_undegraded":"2023-12-04T05:39:18.195035+0000","last_fullsized":"2023-12-04T05:39:18.195035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:53.681504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519109+0000","last_change":"2023-12-04T05:39:17.216228+0000","last_active":"2023-12-04T05:39:18.519109+0000","last_peered":"2023-12-04T05:39:18.519109+0000","last_clean":"2023-12-04T05:39:18.519109+0000","last_became_active":"2023-12-04T05:39:17.215958+0000","last_became_peered":"2023-12-04T05:39:17.215958+0000","last_unstale":"2023-12-04T05:39:18.519109+0000","last_undegraded":"2023-12-04T05:39:18.519109+0000","last_fullsized":"2023-12-04T05:39:18.519109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495336+0000","last_change":"2023-12-04T05:39:17.212872+0000","last_active":"2023-12-04T05:39:18.495336+0000","last_peered":"2023-12-04T05:39:18.495336+0000","last_clean":"2023-12-04T05:39:18.495336+0000","last_became_active":"2023-12-04T05:39:17.212550+0000","last_became_peered":"2023-12-04T05:39:17.212550+0000","last_unstale":"2023-12-04T05:39:18.495336+0000","last_undegraded":"2023-12-04T05:39:18.495336+0000","last_fullsized":"2023-12-04T05:39:18.495336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519658+0000","last_change":"2023-12-04T05:39:17.210290+0000","last_active":"2023-12-04T05:39:18.519658+0000","last_peered":"2023-12-04T05:39:18.519658+0000","last_clean":"2023-12-04T05:39:18.519658+0000","last_became_active":"2023-12-04T05:39:17.210041+0000","last_became_peered":"2023-12-04T05:39:17.210041+0000","last_unstale":"2023-12-04T05:39:18.519658+0000","last_undegraded":"2023-12-04T05:39:18.519658+0000","last_fullsized":"2023-12-04T05:39:18.519658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:01:33.532203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519694+0000","last_change":"2023-12-04T05:39:17.212608+0000","last_active":"2023-12-04T05:39:18.519694+0000","last_peered":"2023-12-04T05:39:18.519694+0000","last_clean":"2023-12-04T05:39:18.519694+0000","last_became_active":"2023-12-04T05:39:17.212249+0000","last_became_peered":"2023-12-04T05:39:17.212249+0000","last_unstale":"2023-12-04T05:39:18.519694+0000","last_undegraded":"2023-12-04T05:39:18.519694+0000","last_fullsized":"2023-12-04T05:39:18.519694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519356+0000","last_change":"2023-12-04T05:39:17.206680+0000","last_active":"2023-12-04T05:39:18.519356+0000","last_peered":"2023-12-04T05:39:18.519356+0000","last_clean":"2023-12-04T05:39:18.519356+0000","last_became_active":"2023-12-04T05:39:17.206326+0000","last_became_peered":"2023-12-04T05:39:17.206326+0000","last_unstale":"2023-12-04T05:39:18.519356+0000","last_undegraded":"2023-12-04T05:39:18.519356+0000","last_fullsized":"2023-12-04T05:39:18.519356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194773+0000","last_change":"2023-12-04T05:39:17.206360+0000","last_active":"2023-12-04T05:39:18.194773+0000","last_peered":"2023-12-04T05:39:18.194773+0000","last_clean":"2023-12-04T05:39:18.194773+0000","last_became_active":"2023-12-04T05:39:17.205989+0000","last_became_peered":"2023-12-04T05:39:17.205989+0000","last_unstale":"2023-12-04T05:39:18.194773+0000","last_undegraded":"2023-12-04T05:39:18.194773+0000","last_fullsized":"2023-12-04T05:39:18.194773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:22:45.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495552+0000","last_change":"2023-12-04T05:39:17.207638+0000","last_active":"2023-12-04T05:39:18.495552+0000","last_peered":"2023-12-04T05:39:18.495552+0000","last_clean":"2023-12-04T05:39:18.495552+0000","last_became_active":"2023-12-04T05:39:17.207368+0000","last_became_peered":"2023-12-04T05:39:17.207368+0000","last_unstale":"2023-12-04T05:39:18.495552+0000","last_undegraded":"2023-12-04T05:39:18.495552+0000","last_fullsized":"2023-12-04T05:39:18.495552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518614+0000","last_change":"2023-12-04T05:39:17.210284+0000","last_active":"2023-12-04T05:39:18.518614+0000","last_peered":"2023-12-04T05:39:18.518614+0000","last_clean":"2023-12-04T05:39:18.518614+0000","last_became_active":"2023-12-04T05:39:17.209989+0000","last_became_peered":"2023-12-04T05:39:17.209989+0000","last_unstale":"2023-12-04T05:39:18.518614+0000","last_undegraded":"2023-12-04T05:39:18.518614+0000","last_fullsized":"2023-12-04T05:39:18.518614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195553+0000","last_change":"2023-12-04T05:39:17.207109+0000","last_active":"2023-12-04T05:39:18.195553+0000","last_peered":"2023-12-04T05:39:18.195553+0000","last_clean":"2023-12-04T05:39:18.195553+0000","last_became_active":"2023-12-04T05:39:17.205722+0000","last_became_peered":"2023-12-04T05:39:17.205722+0000","last_unstale":"2023-12-04T05:39:18.195553+0000","last_undegraded":"2023-12-04T05:39:18.195553+0000","last_fullsized":"2023-12-04T05:39:18.195553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194533+0000","last_change":"2023-12-04T05:39:17.213549+0000","last_active":"2023-12-04T05:39:18.194533+0000","last_peered":"2023-12-04T05:39:18.194533+0000","last_clean":"2023-12-04T05:39:18.194533+0000","last_became_active":"2023-12-04T05:39:17.212758+0000","last_became_peered":"2023-12-04T05:39:17.212758+0000","last_unstale":"2023-12-04T05:39:18.194533+0000","last_undegraded":"2023-12-04T05:39:18.194533+0000","last_fullsized":"2023-12-04T05:39:18.194533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195513+0000","last_change":"2023-12-04T05:39:17.210092+0000","last_active":"2023-12-04T05:39:18.195513+0000","last_peered":"2023-12-04T05:39:18.195513+0000","last_clean":"2023-12-04T05:39:18.195513+0000","last_became_active":"2023-12-04T05:39:17.209828+0000","last_became_peered":"2023-12-04T05:39:17.209828+0000","last_unstale":"2023-12-04T05:39:18.195513+0000","last_undegraded":"2023-12-04T05:39:18.195513+0000","last_fullsized":"2023-12-04T05:39:18.195513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:01:33.532203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518749+0000","last_change":"2023-12-04T05:39:17.206756+0000","last_active":"2023-12-04T05:39:18.518749+0000","last_peered":"2023-12-04T05:39:18.518749+0000","last_clean":"2023-12-04T05:39:18.518749+0000","last_became_active":"2023-12-04T05:39:17.206499+0000","last_became_peered":"2023-12-04T05:39:17.206499+0000","last_unstale":"2023-12-04T05:39:18.518749+0000","last_undegraded":"2023-12-04T05:39:18.518749+0000","last_fullsized":"2023-12-04T05:39:18.518749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:33.381391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519868+0000","last_change":"2023-12-04T05:39:17.213693+0000","last_active":"2023-12-04T05:39:18.519868+0000","last_peered":"2023-12-04T05:39:18.519868+0000","last_clean":"2023-12-04T05:39:18.519868+0000","last_became_active":"2023-12-04T05:39:17.213404+0000","last_became_peered":"2023-12-04T05:39:17.213404+0000","last_unstale":"2023-12-04T05:39:18.519868+0000","last_undegraded":"2023-12-04T05:39:18.519868+0000","last_fullsized":"2023-12-04T05:39:18.519868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:27:18.604272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519451+0000","last_change":"2023-12-04T05:39:17.211735+0000","last_active":"2023-12-04T05:39:18.519451+0000","last_peered":"2023-12-04T05:39:18.519451+0000","last_clean":"2023-12-04T05:39:18.519451+0000","last_became_active":"2023-12-04T05:39:17.211450+0000","last_became_peered":"2023-12-04T05:39:17.211450+0000","last_unstale":"2023-12-04T05:39:18.519451+0000","last_undegraded":"2023-12-04T05:39:18.519451+0000","last_fullsized":"2023-12-04T05:39:18.519451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518422+0000","last_change":"2023-12-04T05:39:17.215901+0000","last_active":"2023-12-04T05:39:18.518422+0000","last_peered":"2023-12-04T05:39:18.518422+0000","last_clean":"2023-12-04T05:39:18.518422+0000","last_became_active":"2023-12-04T05:39:17.215395+0000","last_became_peered":"2023-12-04T05:39:17.215395+0000","last_unstale":"2023-12-04T05:39:18.518422+0000","last_undegraded":"2023-12-04T05:39:18.518422+0000","last_fullsized":"2023-12-04T05:39:18.518422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:01.587525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195031+0000","last_change":"2023-12-04T05:39:17.211134+0000","last_active":"2023-12-04T05:39:18.195031+0000","last_peered":"2023-12-04T05:39:18.195031+0000","last_clean":"2023-12-04T05:39:18.195031+0000","last_became_active":"2023-12-04T05:39:17.210734+0000","last_became_peered":"2023-12-04T05:39:17.210734+0000","last_unstale":"2023-12-04T05:39:18.195031+0000","last_undegraded":"2023-12-04T05:39:18.195031+0000","last_fullsized":"2023-12-04T05:39:18.195031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495425+0000","last_change":"2023-12-04T05:39:17.211310+0000","last_active":"2023-12-04T05:39:18.495425+0000","last_peered":"2023-12-04T05:39:18.495425+0000","last_clean":"2023-12-04T05:39:18.495425+0000","last_became_active":"2023-12-04T05:39:17.210967+0000","last_became_peered":"2023-12-04T05:39:17.210967+0000","last_unstale":"2023-12-04T05:39:18.495425+0000","last_undegraded":"2023-12-04T05:39:18.495425+0000","last_fullsized":"2023-12-04T05:39:18.495425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194980+0000","last_change":"2023-12-04T05:39:17.212493+0000","last_active":"2023-12-04T05:39:18.194980+0000","last_peered":"2023-12-04T05:39:18.194980+0000","last_clean":"2023-12-04T05:39:18.194980+0000","last_became_active":"2023-12-04T05:39:17.212240+0000","last_became_peered":"2023-12-04T05:39:17.212240+0000","last_unstale":"2023-12-04T05:39:18.194980+0000","last_undegraded":"2023-12-04T05:39:18.194980+0000","last_fullsized":"2023-12-04T05:39:18.194980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495743+0000","last_change":"2023-12-04T05:39:17.211088+0000","last_active":"2023-12-04T05:39:18.495743+0000","last_peered":"2023-12-04T05:39:18.495743+0000","last_clean":"2023-12-04T05:39:18.495743+0000","last_became_active":"2023-12-04T05:39:17.210779+0000","last_became_peered":"2023-12-04T05:39:17.210779+0000","last_unstale":"2023-12-04T05:39:18.495743+0000","last_undegraded":"2023-12-04T05:39:18.495743+0000","last_fullsized":"2023-12-04T05:39:18.495743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519676+0000","last_change":"2023-12-04T05:39:17.213404+0000","last_active":"2023-12-04T05:39:18.519676+0000","last_peered":"2023-12-04T05:39:18.519676+0000","last_clean":"2023-12-04T05:39:18.519676+0000","last_became_active":"2023-12-04T05:39:17.213153+0000","last_became_peered":"2023-12-04T05:39:17.213153+0000","last_unstale":"2023-12-04T05:39:18.519676+0000","last_undegraded":"2023-12-04T05:39:18.519676+0000","last_fullsized":"2023-12-04T05:39:18.519676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195771+0000","last_change":"2023-12-04T05:39:17.207997+0000","last_active":"2023-12-04T05:39:18.195771+0000","last_peered":"2023-12-04T05:39:18.195771+0000","last_clean":"2023-12-04T05:39:18.195771+0000","last_became_active":"2023-12-04T05:39:17.207537+0000","last_became_peered":"2023-12-04T05:39:17.207537+0000","last_unstale":"2023-12-04T05:39:18.195771+0000","last_undegraded":"2023-12-04T05:39:18.195771+0000","last_fullsized":"2023-12-04T05:39:18.195771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495833+0000","last_change":"2023-12-04T05:39:17.211215+0000","last_active":"2023-12-04T05:39:18.495833+0000","last_peered":"2023-12-04T05:39:18.495833+0000","last_clean":"2023-12-04T05:39:18.495833+0000","last_became_active":"2023-12-04T05:39:17.210930+0000","last_became_peered":"2023-12-04T05:39:17.210930+0000","last_unstale":"2023-12-04T05:39:18.495833+0000","last_undegraded":"2023-12-04T05:39:18.495833+0000","last_fullsized":"2023-12-04T05:39:18.495833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518602+0000","last_change":"2023-12-04T05:39:17.215388+0000","last_active":"2023-12-04T05:39:18.518602+0000","last_peered":"2023-12-04T05:39:18.518602+0000","last_clean":"2023-12-04T05:39:18.518602+0000","last_became_active":"2023-12-04T05:39:17.215082+0000","last_became_peered":"2023-12-04T05:39:17.215082+0000","last_unstale":"2023-12-04T05:39:18.518602+0000","last_undegraded":"2023-12-04T05:39:18.518602+0000","last_fullsized":"2023-12-04T05:39:18.518602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195233+0000","last_change":"2023-12-04T05:39:17.207162+0000","last_active":"2023-12-04T05:39:18.195233+0000","last_peered":"2023-12-04T05:39:18.195233+0000","last_clean":"2023-12-04T05:39:18.195233+0000","last_became_active":"2023-12-04T05:39:17.206737+0000","last_became_peered":"2023-12-04T05:39:17.206737+0000","last_unstale":"2023-12-04T05:39:18.195233+0000","last_undegraded":"2023-12-04T05:39:18.195233+0000","last_fullsized":"2023-12-04T05:39:18.195233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518154+0000","last_change":"2023-12-04T05:39:17.204908+0000","last_active":"2023-12-04T05:39:18.518154+0000","last_peered":"2023-12-04T05:39:18.518154+0000","last_clean":"2023-12-04T05:39:18.518154+0000","last_became_active":"2023-12-04T05:39:17.204656+0000","last_became_peered":"2023-12-04T05:39:17.204656+0000","last_unstale":"2023-12-04T05:39:18.518154+0000","last_undegraded":"2023-12-04T05:39:18.518154+0000","last_fullsized":"2023-12-04T05:39:18.518154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:33.059240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518987+0000","last_change":"2023-12-04T05:39:17.211113+0000","last_active":"2023-12-04T05:39:18.518987+0000","last_peered":"2023-12-04T05:39:18.518987+0000","last_clean":"2023-12-04T05:39:18.518987+0000","last_became_active":"2023-12-04T05:39:17.210626+0000","last_became_peered":"2023-12-04T05:39:17.210626+0000","last_unstale":"2023-12-04T05:39:18.518987+0000","last_undegraded":"2023-12-04T05:39:18.518987+0000","last_fullsized":"2023-12-04T05:39:18.518987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194380+0000","last_change":"2023-12-04T05:39:17.207261+0000","last_active":"2023-12-04T05:39:18.194380+0000","last_peered":"2023-12-04T05:39:18.194380+0000","last_clean":"2023-12-04T05:39:18.194380+0000","last_became_active":"2023-12-04T05:39:17.206957+0000","last_became_peered":"2023-12-04T05:39:17.206957+0000","last_unstale":"2023-12-04T05:39:18.194380+0000","last_undegraded":"2023-12-04T05:39:18.194380+0000","last_fullsized":"2023-12-04T05:39:18.194380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:38:49.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495637+0000","last_change":"2023-12-04T05:39:16.222472+0000","last_active":"2023-12-04T05:39:18.495637+0000","last_peered":"2023-12-04T05:39:18.495637+0000","last_clean":"2023-12-04T05:39:18.495637+0000","last_became_active":"2023-12-04T05:39:16.222136+0000","last_became_peered":"2023-12-04T05:39:16.222136+0000","last_unstale":"2023-12-04T05:39:18.495637+0000","last_undegraded":"2023-12-04T05:39:18.495637+0000","last_fullsized":"2023-12-04T05:39:18.495637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518714+0000","last_change":"2023-12-04T05:39:17.215145+0000","last_active":"2023-12-04T05:39:18.518714+0000","last_peered":"2023-12-04T05:39:18.518714+0000","last_clean":"2023-12-04T05:39:18.518714+0000","last_became_active":"2023-12-04T05:39:17.214877+0000","last_became_peered":"2023-12-04T05:39:17.214877+0000","last_unstale":"2023-12-04T05:39:18.518714+0000","last_undegraded":"2023-12-04T05:39:18.518714+0000","last_fullsized":"2023-12-04T05:39:18.518714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519318+0000","last_change":"2023-12-04T05:39:16.216663+0000","last_active":"2023-12-04T05:39:18.519318+0000","last_peered":"2023-12-04T05:39:18.519318+0000","last_clean":"2023-12-04T05:39:18.519318+0000","last_became_active":"2023-12-04T05:39:16.216408+0000","last_became_peered":"2023-12-04T05:39:16.216408+0000","last_unstale":"2023-12-04T05:39:18.519318+0000","last_undegraded":"2023-12-04T05:39:18.519318+0000","last_fullsized":"2023-12-04T05:39:18.519318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:01:24.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495399+0000","last_change":"2023-12-04T05:39:16.216780+0000","last_active":"2023-12-04T05:39:18.495399+0000","last_peered":"2023-12-04T05:39:18.495399+0000","last_clean":"2023-12-04T05:39:18.495399+0000","last_became_active":"2023-12-04T05:39:16.216486+0000","last_became_peered":"2023-12-04T05:39:16.216486+0000","last_unstale":"2023-12-04T05:39:18.495399+0000","last_undegraded":"2023-12-04T05:39:18.495399+0000","last_fullsized":"2023-12-04T05:39:18.495399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:00.582466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495379+0000","last_change":"2023-12-04T05:39:17.215850+0000","last_active":"2023-12-04T05:39:18.495379+0000","last_peered":"2023-12-04T05:39:18.495379+0000","last_clean":"2023-12-04T05:39:18.495379+0000","last_became_active":"2023-12-04T05:39:17.215557+0000","last_became_peered":"2023-12-04T05:39:17.215557+0000","last_unstale":"2023-12-04T05:39:18.495379+0000","last_undegraded":"2023-12-04T05:39:18.495379+0000","last_fullsized":"2023-12-04T05:39:18.495379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194854+0000","last_change":"2023-12-04T05:39:16.213564+0000","last_active":"2023-12-04T05:39:18.194854+0000","last_peered":"2023-12-04T05:39:18.194854+0000","last_clean":"2023-12-04T05:39:18.194854+0000","last_became_active":"2023-12-04T05:39:16.213288+0000","last_became_peered":"2023-12-04T05:39:16.213288+0000","last_unstale":"2023-12-04T05:39:18.194854+0000","last_undegraded":"2023-12-04T05:39:18.194854+0000","last_fullsized":"2023-12-04T05:39:18.194854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195514+0000","last_change":"2023-12-04T05:39:17.208143+0000","last_active":"2023-12-04T05:39:18.195514+0000","last_peered":"2023-12-04T05:39:18.195514+0000","last_clean":"2023-12-04T05:39:18.195514+0000","last_became_active":"2023-12-04T05:39:17.207883+0000","last_became_peered":"2023-12-04T05:39:17.207883+0000","last_unstale":"2023-12-04T05:39:18.195514+0000","last_undegraded":"2023-12-04T05:39:18.195514+0000","last_fullsized":"2023-12-04T05:39:18.195514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519244+0000","last_change":"2023-12-04T05:39:16.217144+0000","last_active":"2023-12-04T05:39:18.519244+0000","last_peered":"2023-12-04T05:39:18.519244+0000","last_clean":"2023-12-04T05:39:18.519244+0000","last_became_active":"2023-12-04T05:39:16.216759+0000","last_became_peered":"2023-12-04T05:39:16.216759+0000","last_unstale":"2023-12-04T05:39:18.519244+0000","last_undegraded":"2023-12-04T05:39:18.519244+0000","last_fullsized":"2023-12-04T05:39:18.519244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:22:44.119471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195253+0000","last_change":"2023-12-04T05:39:17.211977+0000","last_active":"2023-12-04T05:39:18.195253+0000","last_peered":"2023-12-04T05:39:18.195253+0000","last_clean":"2023-12-04T05:39:18.195253+0000","last_became_active":"2023-12-04T05:39:17.211707+0000","last_became_peered":"2023-12-04T05:39:17.211707+0000","last_unstale":"2023-12-04T05:39:18.195253+0000","last_undegraded":"2023-12-04T05:39:18.195253+0000","last_fullsized":"2023-12-04T05:39:18.195253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195229+0000","last_change":"2023-12-04T05:39:16.212865+0000","last_active":"2023-12-04T05:39:18.195229+0000","last_peered":"2023-12-04T05:39:18.195229+0000","last_clean":"2023-12-04T05:39:18.195229+0000","last_became_active":"2023-12-04T05:39:16.212574+0000","last_became_peered":"2023-12-04T05:39:16.212574+0000","last_unstale":"2023-12-04T05:39:18.195229+0000","last_undegraded":"2023-12-04T05:39:18.195229+0000","last_fullsized":"2023-12-04T05:39:18.195229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:17.975393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518554+0000","last_change":"2023-12-04T05:39:17.210657+0000","last_active":"2023-12-04T05:39:18.518554+0000","last_peered":"2023-12-04T05:39:18.518554+0000","last_clean":"2023-12-04T05:39:18.518554+0000","last_became_active":"2023-12-04T05:39:17.210197+0000","last_became_peered":"2023-12-04T05:39:17.210197+0000","last_unstale":"2023-12-04T05:39:18.518554+0000","last_undegraded":"2023-12-04T05:39:18.518554+0000","last_fullsized":"2023-12-04T05:39:18.518554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195730+0000","last_change":"2023-12-04T05:39:16.219612+0000","last_active":"2023-12-04T05:39:18.195730+0000","last_peered":"2023-12-04T05:39:18.195730+0000","last_clean":"2023-12-04T05:39:18.195730+0000","last_became_active":"2023-12-04T05:39:16.219305+0000","last_became_peered":"2023-12-04T05:39:16.219305+0000","last_unstale":"2023-12-04T05:39:18.195730+0000","last_undegraded":"2023-12-04T05:39:18.195730+0000","last_fullsized":"2023-12-04T05:39:18.195730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194796+0000","last_change":"2023-12-04T05:39:17.202594+0000","last_active":"2023-12-04T05:39:18.194796+0000","last_peered":"2023-12-04T05:39:18.194796+0000","last_clean":"2023-12-04T05:39:18.194796+0000","last_became_active":"2023-12-04T05:39:17.202079+0000","last_became_peered":"2023-12-04T05:39:17.202079+0000","last_unstale":"2023-12-04T05:39:18.194796+0000","last_undegraded":"2023-12-04T05:39:18.194796+0000","last_fullsized":"2023-12-04T05:39:18.194796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:33.059240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519559+0000","last_change":"2023-12-04T05:39:16.212413+0000","last_active":"2023-12-04T05:39:18.519559+0000","last_peered":"2023-12-04T05:39:18.519559+0000","last_clean":"2023-12-04T05:39:18.519559+0000","last_became_active":"2023-12-04T05:39:16.212127+0000","last_became_peered":"2023-12-04T05:39:16.212127+0000","last_unstale":"2023-12-04T05:39:18.519559+0000","last_undegraded":"2023-12-04T05:39:18.519559+0000","last_fullsized":"2023-12-04T05:39:18.519559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519613+0000","last_change":"2023-12-04T05:39:17.208325+0000","last_active":"2023-12-04T05:39:18.519613+0000","last_peered":"2023-12-04T05:39:18.519613+0000","last_clean":"2023-12-04T05:39:18.519613+0000","last_became_active":"2023-12-04T05:39:17.208060+0000","last_became_peered":"2023-12-04T05:39:17.208060+0000","last_unstale":"2023-12-04T05:39:18.519613+0000","last_undegraded":"2023-12-04T05:39:18.519613+0000","last_fullsized":"2023-12-04T05:39:18.519613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:56:49.040036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518855+0000","last_change":"2023-12-04T05:39:16.217018+0000","last_active":"2023-12-04T05:39:18.518855+0000","last_peered":"2023-12-04T05:39:18.518855+0000","last_clean":"2023-12-04T05:39:18.518855+0000","last_became_active":"2023-12-04T05:39:16.216772+0000","last_became_peered":"2023-12-04T05:39:16.216772+0000","last_unstale":"2023-12-04T05:39:18.518855+0000","last_undegraded":"2023-12-04T05:39:18.518855+0000","last_fullsized":"2023-12-04T05:39:18.518855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:32.054181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518705+0000","last_change":"2023-12-04T05:39:17.213778+0000","last_active":"2023-12-04T05:39:18.518705+0000","last_peered":"2023-12-04T05:39:18.518705+0000","last_clean":"2023-12-04T05:39:18.518705+0000","last_became_active":"2023-12-04T05:39:17.213520+0000","last_became_peered":"2023-12-04T05:39:17.213520+0000","last_unstale":"2023-12-04T05:39:18.518705+0000","last_undegraded":"2023-12-04T05:39:18.518705+0000","last_fullsized":"2023-12-04T05:39:18.518705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195291+0000","last_change":"2023-12-04T05:39:16.212009+0000","last_active":"2023-12-04T05:39:18.195291+0000","last_peered":"2023-12-04T05:39:18.195291+0000","last_clean":"2023-12-04T05:39:18.195291+0000","last_became_active":"2023-12-04T05:39:16.211611+0000","last_became_peered":"2023-12-04T05:39:16.211611+0000","last_unstale":"2023-12-04T05:39:18.195291+0000","last_undegraded":"2023-12-04T05:39:18.195291+0000","last_fullsized":"2023-12-04T05:39:18.195291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:24.393125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519548+0000","last_change":"2023-12-04T05:39:17.210008+0000","last_active":"2023-12-04T05:39:18.519548+0000","last_peered":"2023-12-04T05:39:18.519548+0000","last_clean":"2023-12-04T05:39:18.519548+0000","last_became_active":"2023-12-04T05:39:17.209767+0000","last_became_peered":"2023-12-04T05:39:17.209767+0000","last_unstale":"2023-12-04T05:39:18.519548+0000","last_undegraded":"2023-12-04T05:39:18.519548+0000","last_fullsized":"2023-12-04T05:39:18.519548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:35:27.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519053+0000","last_change":"2023-12-04T05:39:16.219504+0000","last_active":"2023-12-04T05:39:18.519053+0000","last_peered":"2023-12-04T05:39:18.519053+0000","last_clean":"2023-12-04T05:39:18.519053+0000","last_became_active":"2023-12-04T05:39:16.219241+0000","last_became_peered":"2023-12-04T05:39:16.219241+0000","last_unstale":"2023-12-04T05:39:18.519053+0000","last_undegraded":"2023-12-04T05:39:18.519053+0000","last_fullsized":"2023-12-04T05:39:18.519053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.913440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519043+0000","last_change":"2023-12-04T05:39:17.210165+0000","last_active":"2023-12-04T05:39:18.519043+0000","last_peered":"2023-12-04T05:39:18.519043+0000","last_clean":"2023-12-04T05:39:18.519043+0000","last_became_active":"2023-12-04T05:39:17.209917+0000","last_became_peered":"2023-12-04T05:39:17.209917+0000","last_unstale":"2023-12-04T05:39:18.519043+0000","last_undegraded":"2023-12-04T05:39:18.519043+0000","last_fullsized":"2023-12-04T05:39:18.519043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518077+0000","last_change":"2023-12-04T05:39:16.216340+0000","last_active":"2023-12-04T05:39:18.518077+0000","last_peered":"2023-12-04T05:39:18.518077+0000","last_clean":"2023-12-04T05:39:18.518077+0000","last_became_active":"2023-12-04T05:39:16.216034+0000","last_became_peered":"2023-12-04T05:39:16.216034+0000","last_unstale":"2023-12-04T05:39:18.518077+0000","last_undegraded":"2023-12-04T05:39:18.518077+0000","last_fullsized":"2023-12-04T05:39:18.518077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.913440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519028+0000","last_change":"2023-12-04T05:39:17.216087+0000","last_active":"2023-12-04T05:39:18.519028+0000","last_peered":"2023-12-04T05:39:18.519028+0000","last_clean":"2023-12-04T05:39:18.519028+0000","last_became_active":"2023-12-04T05:39:17.215420+0000","last_became_peered":"2023-12-04T05:39:17.215420+0000","last_unstale":"2023-12-04T05:39:18.519028+0000","last_undegraded":"2023-12-04T05:39:18.519028+0000","last_fullsized":"2023-12-04T05:39:18.519028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194623+0000","last_change":"2023-12-04T05:39:16.218872+0000","last_active":"2023-12-04T05:39:18.194623+0000","last_peered":"2023-12-04T05:39:18.194623+0000","last_clean":"2023-12-04T05:39:18.194623+0000","last_became_active":"2023-12-04T05:39:16.218609+0000","last_became_peered":"2023-12-04T05:39:16.218609+0000","last_unstale":"2023-12-04T05:39:18.194623+0000","last_undegraded":"2023-12-04T05:39:18.194623+0000","last_fullsized":"2023-12-04T05:39:18.194623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T05:42:48.541211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519439+0000","last_change":"2023-12-04T05:39:17.209465+0000","last_active":"2023-12-04T05:39:18.519439+0000","last_peered":"2023-12-04T05:39:18.519439+0000","last_clean":"2023-12-04T05:39:18.519439+0000","last_became_active":"2023-12-04T05:39:17.209210+0000","last_became_peered":"2023-12-04T05:39:17.209210+0000","last_unstale":"2023-12-04T05:39:18.519439+0000","last_undegraded":"2023-12-04T05:39:18.519439+0000","last_fullsized":"2023-12-04T05:39:18.519439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518796+0000","last_change":"2023-12-04T05:39:16.212731+0000","last_active":"2023-12-04T05:39:18.518796+0000","last_peered":"2023-12-04T05:39:18.518796+0000","last_clean":"2023-12-04T05:39:18.518796+0000","last_became_active":"2023-12-04T05:39:16.212398+0000","last_became_peered":"2023-12-04T05:39:16.212398+0000","last_unstale":"2023-12-04T05:39:18.518796+0000","last_undegraded":"2023-12-04T05:39:18.518796+0000","last_fullsized":"2023-12-04T05:39:18.518796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:13.519031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519792+0000","last_change":"2023-12-04T05:39:17.215878+0000","last_active":"2023-12-04T05:39:18.519792+0000","last_peered":"2023-12-04T05:39:18.519792+0000","last_clean":"2023-12-04T05:39:18.519792+0000","last_became_active":"2023-12-04T05:39:17.215600+0000","last_became_peered":"2023-12-04T05:39:17.215600+0000","last_unstale":"2023-12-04T05:39:18.519792+0000","last_undegraded":"2023-12-04T05:39:18.519792+0000","last_fullsized":"2023-12-04T05:39:18.519792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:21:50.865362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195801+0000","last_change":"2023-12-04T05:39:16.219401+0000","last_active":"2023-12-04T05:39:18.195801+0000","last_peered":"2023-12-04T05:39:18.195801+0000","last_clean":"2023-12-04T05:39:18.195801+0000","last_became_active":"2023-12-04T05:39:16.219096+0000","last_became_peered":"2023-12-04T05:39:16.219096+0000","last_unstale":"2023-12-04T05:39:18.195801+0000","last_undegraded":"2023-12-04T05:39:18.195801+0000","last_fullsized":"2023-12-04T05:39:18.195801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195394+0000","last_change":"2023-12-04T05:39:17.208978+0000","last_active":"2023-12-04T05:39:18.195394+0000","last_peered":"2023-12-04T05:39:18.195394+0000","last_clean":"2023-12-04T05:39:18.195394+0000","last_became_active":"2023-12-04T05:39:17.208163+0000","last_became_peered":"2023-12-04T05:39:17.208163+0000","last_unstale":"2023-12-04T05:39:18.195394+0000","last_undegraded":"2023-12-04T05:39:18.195394+0000","last_fullsized":"2023-12-04T05:39:18.195394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:35:27.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518395+0000","last_change":"2023-12-04T05:39:16.216610+0000","last_active":"2023-12-04T05:39:18.518395+0000","last_peered":"2023-12-04T05:39:18.518395+0000","last_clean":"2023-12-04T05:39:18.518395+0000","last_became_active":"2023-12-04T05:39:16.216343+0000","last_became_peered":"2023-12-04T05:39:16.216343+0000","last_unstale":"2023-12-04T05:39:18.518395+0000","last_undegraded":"2023-12-04T05:39:18.518395+0000","last_fullsized":"2023-12-04T05:39:18.518395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:06:28.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519731+0000","last_change":"2023-12-04T05:39:17.206092+0000","last_active":"2023-12-04T05:39:18.519731+0000","last_peered":"2023-12-04T05:39:18.519731+0000","last_clean":"2023-12-04T05:39:18.519731+0000","last_became_active":"2023-12-04T05:39:17.205430+0000","last_became_peered":"2023-12-04T05:39:17.205430+0000","last_unstale":"2023-12-04T05:39:18.519731+0000","last_undegraded":"2023-12-04T05:39:18.519731+0000","last_fullsized":"2023-12-04T05:39:18.519731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:28:25.883452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-04T05:39:17.200250+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:39:17.200250+0000","last_peered":"2023-12-04T05:39:17.200250+0000","last_clean":"2023-12-04T05:39:17.200250+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:39:17.200250+0000","last_undegraded":"2023-12-04T05:39:17.200250+0000","last_fullsized":"2023-12-04T05:39:17.200250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194896+0000","last_change":"2023-12-04T05:39:16.211697+0000","last_active":"2023-12-04T05:39:18.194896+0000","last_peered":"2023-12-04T05:39:18.194896+0000","last_clean":"2023-12-04T05:39:18.194896+0000","last_became_active":"2023-12-04T05:39:16.211421+0000","last_became_peered":"2023-12-04T05:39:16.211421+0000","last_unstale":"2023-12-04T05:39:18.194896+0000","last_undegraded":"2023-12-04T05:39:18.194896+0000","last_fullsized":"2023-12-04T05:39:18.194896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:05.599778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495509+0000","last_change":"2023-12-04T05:39:17.212309+0000","last_active":"2023-12-04T05:39:18.495509+0000","last_peered":"2023-12-04T05:39:18.495509+0000","last_clean":"2023-12-04T05:39:18.495509+0000","last_became_active":"2023-12-04T05:39:17.212069+0000","last_became_peered":"2023-12-04T05:39:17.212069+0000","last_unstale":"2023-12-04T05:39:18.495509+0000","last_undegraded":"2023-12-04T05:39:18.495509+0000","last_fullsized":"2023-12-04T05:39:18.495509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518862+0000","last_change":"2023-12-04T05:39:16.222456+0000","last_active":"2023-12-04T05:39:18.518862+0000","last_peered":"2023-12-04T05:39:18.518862+0000","last_clean":"2023-12-04T05:39:18.518862+0000","last_became_active":"2023-12-04T05:39:16.222007+0000","last_became_peered":"2023-12-04T05:39:16.222007+0000","last_unstale":"2023-12-04T05:39:18.518862+0000","last_undegraded":"2023-12-04T05:39:18.518862+0000","last_fullsized":"2023-12-04T05:39:18.518862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518357+0000","last_change":"2023-12-04T05:39:17.209347+0000","last_active":"2023-12-04T05:39:18.518357+0000","last_peered":"2023-12-04T05:39:18.518357+0000","last_clean":"2023-12-04T05:39:18.518357+0000","last_became_active":"2023-12-04T05:39:17.209044+0000","last_became_peered":"2023-12-04T05:39:17.209044+0000","last_unstale":"2023-12-04T05:39:18.518357+0000","last_undegraded":"2023-12-04T05:39:18.518357+0000","last_fullsized":"2023-12-04T05:39:18.518357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:40.539141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518478+0000","last_change":"2023-12-04T05:39:16.213448+0000","last_active":"2023-12-04T05:39:18.518478+0000","last_peered":"2023-12-04T05:39:18.518478+0000","last_clean":"2023-12-04T05:39:18.518478+0000","last_became_active":"2023-12-04T05:39:16.212426+0000","last_became_peered":"2023-12-04T05:39:16.212426+0000","last_unstale":"2023-12-04T05:39:18.518478+0000","last_undegraded":"2023-12-04T05:39:18.518478+0000","last_fullsized":"2023-12-04T05:39:18.518478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:00:49.392004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195538+0000","last_change":"2023-12-04T05:39:17.206807+0000","last_active":"2023-12-04T05:39:18.195538+0000","last_peered":"2023-12-04T05:39:18.195538+0000","last_clean":"2023-12-04T05:39:18.195538+0000","last_became_active":"2023-12-04T05:39:17.206502+0000","last_became_peered":"2023-12-04T05:39:17.206502+0000","last_unstale":"2023-12-04T05:39:18.195538+0000","last_undegraded":"2023-12-04T05:39:18.195538+0000","last_fullsized":"2023-12-04T05:39:18.195538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195374+0000","last_change":"2023-12-04T05:39:16.216126+0000","last_active":"2023-12-04T05:39:18.195374+0000","last_peered":"2023-12-04T05:39:18.195374+0000","last_clean":"2023-12-04T05:39:18.195374+0000","last_became_active":"2023-12-04T05:39:16.215729+0000","last_became_peered":"2023-12-04T05:39:16.215729+0000","last_unstale":"2023-12-04T05:39:18.195374+0000","last_undegraded":"2023-12-04T05:39:18.195374+0000","last_fullsized":"2023-12-04T05:39:18.195374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:18.392982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194711+0000","last_change":"2023-12-04T05:39:17.215887+0000","last_active":"2023-12-04T05:39:18.194711+0000","last_peered":"2023-12-04T05:39:18.194711+0000","last_clean":"2023-12-04T05:39:18.194711+0000","last_became_active":"2023-12-04T05:39:17.215638+0000","last_became_peered":"2023-12-04T05:39:17.215638+0000","last_unstale":"2023-12-04T05:39:18.194711+0000","last_undegraded":"2023-12-04T05:39:18.194711+0000","last_fullsized":"2023-12-04T05:39:18.194711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:01:25.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195699+0000","last_change":"2023-12-04T05:39:16.222396+0000","last_active":"2023-12-04T05:39:18.195699+0000","last_peered":"2023-12-04T05:39:18.195699+0000","last_clean":"2023-12-04T05:39:18.195699+0000","last_became_active":"2023-12-04T05:39:16.222118+0000","last_became_peered":"2023-12-04T05:39:16.222118+0000","last_unstale":"2023-12-04T05:39:18.195699+0000","last_undegraded":"2023-12-04T05:39:18.195699+0000","last_fullsized":"2023-12-04T05:39:18.195699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195595+0000","last_change":"2023-12-04T05:39:17.211083+0000","last_active":"2023-12-04T05:39:18.195595+0000","last_peered":"2023-12-04T05:39:18.195595+0000","last_clean":"2023-12-04T05:39:18.195595+0000","last_became_active":"2023-12-04T05:39:17.210751+0000","last_became_peered":"2023-12-04T05:39:17.210751+0000","last_unstale":"2023-12-04T05:39:18.195595+0000","last_undegraded":"2023-12-04T05:39:18.195595+0000","last_fullsized":"2023-12-04T05:39:18.195595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:28:25.883452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519823+0000","last_change":"2023-12-04T05:39:16.222777+0000","last_active":"2023-12-04T05:39:18.519823+0000","last_peered":"2023-12-04T05:39:18.519823+0000","last_clean":"2023-12-04T05:39:18.519823+0000","last_became_active":"2023-12-04T05:39:16.222487+0000","last_became_peered":"2023-12-04T05:39:16.222487+0000","last_unstale":"2023-12-04T05:39:18.519823+0000","last_undegraded":"2023-12-04T05:39:18.519823+0000","last_fullsized":"2023-12-04T05:39:18.519823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:20.634044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195752+0000","last_change":"2023-12-04T05:39:17.206784+0000","last_active":"2023-12-04T05:39:18.195752+0000","last_peered":"2023-12-04T05:39:18.195752+0000","last_clean":"2023-12-04T05:39:18.195752+0000","last_became_active":"2023-12-04T05:39:17.206468+0000","last_became_peered":"2023-12-04T05:39:17.206468+0000","last_unstale":"2023-12-04T05:39:18.195752+0000","last_undegraded":"2023-12-04T05:39:18.195752+0000","last_fullsized":"2023-12-04T05:39:18.195752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195811+0000","last_change":"2023-12-04T05:39:16.219828+0000","last_active":"2023-12-04T05:39:18.195811+0000","last_peered":"2023-12-04T05:39:18.195811+0000","last_clean":"2023-12-04T05:39:18.195811+0000","last_became_active":"2023-12-04T05:39:16.219564+0000","last_became_peered":"2023-12-04T05:39:16.219564+0000","last_unstale":"2023-12-04T05:39:18.195811+0000","last_undegraded":"2023-12-04T05:39:18.195811+0000","last_fullsized":"2023-12-04T05:39:18.195811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:00.582466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495806+0000","last_change":"2023-12-04T05:39:17.213462+0000","last_active":"2023-12-04T05:39:18.495806+0000","last_peered":"2023-12-04T05:39:18.495806+0000","last_clean":"2023-12-04T05:39:18.495806+0000","last_became_active":"2023-12-04T05:39:17.213163+0000","last_became_peered":"2023-12-04T05:39:17.213163+0000","last_unstale":"2023-12-04T05:39:18.495806+0000","last_undegraded":"2023-12-04T05:39:18.495806+0000","last_fullsized":"2023-12-04T05:39:18.495806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519346+0000","last_change":"2023-12-04T05:39:16.225197+0000","last_active":"2023-12-04T05:39:18.519346+0000","last_peered":"2023-12-04T05:39:18.519346+0000","last_clean":"2023-12-04T05:39:18.519346+0000","last_became_active":"2023-12-04T05:39:16.224896+0000","last_became_peered":"2023-12-04T05:39:16.224896+0000","last_unstale":"2023-12-04T05:39:18.519346+0000","last_undegraded":"2023-12-04T05:39:18.519346+0000","last_fullsized":"2023-12-04T05:39:18.519346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518458+0000","last_change":"2023-12-04T05:39:17.211876+0000","last_active":"2023-12-04T05:39:18.518458+0000","last_peered":"2023-12-04T05:39:18.518458+0000","last_clean":"2023-12-04T05:39:18.518458+0000","last_became_active":"2023-12-04T05:39:17.211580+0000","last_became_peered":"2023-12-04T05:39:17.211580+0000","last_unstale":"2023-12-04T05:39:18.518458+0000","last_undegraded":"2023-12-04T05:39:18.518458+0000","last_fullsized":"2023-12-04T05:39:18.518458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518857+0000","last_change":"2023-12-04T05:39:16.222816+0000","last_active":"2023-12-04T05:39:18.518857+0000","last_peered":"2023-12-04T05:39:18.518857+0000","last_clean":"2023-12-04T05:39:18.518857+0000","last_became_active":"2023-12-04T05:39:16.222453+0000","last_became_peered":"2023-12-04T05:39:16.222453+0000","last_unstale":"2023-12-04T05:39:18.518857+0000","last_undegraded":"2023-12-04T05:39:18.518857+0000","last_fullsized":"2023-12-04T05:39:18.518857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194429+0000","last_change":"2023-12-04T05:39:17.204065+0000","last_active":"2023-12-04T05:39:18.194429+0000","last_peered":"2023-12-04T05:39:18.194429+0000","last_clean":"2023-12-04T05:39:18.194429+0000","last_became_active":"2023-12-04T05:39:17.203770+0000","last_became_peered":"2023-12-04T05:39:17.203770+0000","last_unstale":"2023-12-04T05:39:18.194429+0000","last_undegraded":"2023-12-04T05:39:18.194429+0000","last_fullsized":"2023-12-04T05:39:18.194429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:00:50.397064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195575+0000","last_change":"2023-12-04T05:39:16.223148+0000","last_active":"2023-12-04T05:39:18.195575+0000","last_peered":"2023-12-04T05:39:18.195575+0000","last_clean":"2023-12-04T05:39:18.195575+0000","last_became_active":"2023-12-04T05:39:16.222884+0000","last_became_peered":"2023-12-04T05:39:16.222884+0000","last_unstale":"2023-12-04T05:39:18.195575+0000","last_undegraded":"2023-12-04T05:39:18.195575+0000","last_fullsized":"2023-12-04T05:39:18.195575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:32.054181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495446+0000","last_change":"2023-12-04T05:39:17.207953+0000","last_active":"2023-12-04T05:39:18.495446+0000","last_peered":"2023-12-04T05:39:18.495446+0000","last_clean":"2023-12-04T05:39:18.495446+0000","last_became_active":"2023-12-04T05:39:17.207612+0000","last_became_peered":"2023-12-04T05:39:17.207612+0000","last_unstale":"2023-12-04T05:39:18.495446+0000","last_undegraded":"2023-12-04T05:39:18.495446+0000","last_fullsized":"2023-12-04T05:39:18.495446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195660+0000","last_change":"2023-12-04T05:39:16.216886+0000","last_active":"2023-12-04T05:39:18.195660+0000","last_peered":"2023-12-04T05:39:18.195660+0000","last_clean":"2023-12-04T05:39:18.195660+0000","last_became_active":"2023-12-04T05:39:16.215666+0000","last_became_peered":"2023-12-04T05:39:16.215666+0000","last_unstale":"2023-12-04T05:39:18.195660+0000","last_undegraded":"2023-12-04T05:39:18.195660+0000","last_fullsized":"2023-12-04T05:39:18.195660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495633+0000","last_change":"2023-12-04T05:39:17.216210+0000","last_active":"2023-12-04T05:39:18.495633+0000","last_peered":"2023-12-04T05:39:18.495633+0000","last_clean":"2023-12-04T05:39:18.495633+0000","last_became_active":"2023-12-04T05:39:17.215926+0000","last_became_peered":"2023-12-04T05:39:17.215926+0000","last_unstale":"2023-12-04T05:39:18.495633+0000","last_undegraded":"2023-12-04T05:39:18.495633+0000","last_fullsized":"2023-12-04T05:39:18.495633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:53.681504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519432+0000","last_change":"2023-12-04T05:39:16.222353+0000","last_active":"2023-12-04T05:39:18.519432+0000","last_peered":"2023-12-04T05:39:18.519432+0000","last_clean":"2023-12-04T05:39:18.519432+0000","last_became_active":"2023-12-04T05:39:16.221956+0000","last_became_peered":"2023-12-04T05:39:16.221956+0000","last_unstale":"2023-12-04T05:39:18.519432+0000","last_undegraded":"2023-12-04T05:39:18.519432+0000","last_fullsized":"2023-12-04T05:39:18.519432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519216+0000","last_change":"2023-12-04T05:39:17.210884+0000","last_active":"2023-12-04T05:39:18.519216+0000","last_peered":"2023-12-04T05:39:18.519216+0000","last_clean":"2023-12-04T05:39:18.519216+0000","last_became_active":"2023-12-04T05:39:17.210601+0000","last_became_peered":"2023-12-04T05:39:17.210601+0000","last_unstale":"2023-12-04T05:39:18.519216+0000","last_undegraded":"2023-12-04T05:39:18.519216+0000","last_fullsized":"2023-12-04T05:39:18.519216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195487+0000","last_change":"2023-12-04T05:39:17.204261+0000","last_active":"2023-12-04T05:39:18.195487+0000","last_peered":"2023-12-04T05:39:18.195487+0000","last_clean":"2023-12-04T05:39:18.195487+0000","last_became_active":"2023-12-04T05:39:17.203985+0000","last_became_peered":"2023-12-04T05:39:17.203985+0000","last_unstale":"2023-12-04T05:39:18.195487+0000","last_undegraded":"2023-12-04T05:39:18.195487+0000","last_fullsized":"2023-12-04T05:39:18.195487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:56:49.040036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519438+0000","last_change":"2023-12-04T05:39:16.222742+0000","last_active":"2023-12-04T05:39:18.519438+0000","last_peered":"2023-12-04T05:39:18.519438+0000","last_clean":"2023-12-04T05:39:18.519438+0000","last_became_active":"2023-12-04T05:39:16.222073+0000","last_became_peered":"2023-12-04T05:39:16.222073+0000","last_unstale":"2023-12-04T05:39:18.519438+0000","last_undegraded":"2023-12-04T05:39:18.519438+0000","last_fullsized":"2023-12-04T05:39:18.519438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195197+0000","last_change":"2023-12-04T05:39:17.204727+0000","last_active":"2023-12-04T05:39:18.195197+0000","last_peered":"2023-12-04T05:39:18.195197+0000","last_clean":"2023-12-04T05:39:18.195197+0000","last_became_active":"2023-12-04T05:39:17.203997+0000","last_became_peered":"2023-12-04T05:39:17.203997+0000","last_unstale":"2023-12-04T05:39:18.195197+0000","last_undegraded":"2023-12-04T05:39:18.195197+0000","last_fullsized":"2023-12-04T05:39:18.195197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519715+0000","last_change":"2023-12-04T05:39:16.223432+0000","last_active":"2023-12-04T05:39:18.519715+0000","last_peered":"2023-12-04T05:39:18.519715+0000","last_clean":"2023-12-04T05:39:18.519715+0000","last_became_active":"2023-12-04T05:39:16.222966+0000","last_became_peered":"2023-12-04T05:39:16.222966+0000","last_unstale":"2023-12-04T05:39:18.519715+0000","last_undegraded":"2023-12-04T05:39:18.519715+0000","last_fullsized":"2023-12-04T05:39:18.519715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:24.393125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518460+0000","last_change":"2023-12-04T05:39:17.214833+0000","last_active":"2023-12-04T05:39:18.518460+0000","last_peered":"2023-12-04T05:39:18.518460+0000","last_clean":"2023-12-04T05:39:18.518460+0000","last_became_active":"2023-12-04T05:39:17.214570+0000","last_became_peered":"2023-12-04T05:39:17.214570+0000","last_unstale":"2023-12-04T05:39:18.518460+0000","last_undegraded":"2023-12-04T05:39:18.518460+0000","last_fullsized":"2023-12-04T05:39:18.518460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:24.110066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495773+0000","last_change":"2023-12-04T05:39:16.218785+0000","last_active":"2023-12-04T05:39:18.495773+0000","last_peered":"2023-12-04T05:39:18.495773+0000","last_clean":"2023-12-04T05:39:18.495773+0000","last_became_active":"2023-12-04T05:39:16.218480+0000","last_became_peered":"2023-12-04T05:39:16.218480+0000","last_unstale":"2023-12-04T05:39:18.495773+0000","last_undegraded":"2023-12-04T05:39:18.495773+0000","last_fullsized":"2023-12-04T05:39:18.495773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495766+0000","last_change":"2023-12-04T05:39:17.208223+0000","last_active":"2023-12-04T05:39:18.495766+0000","last_peered":"2023-12-04T05:39:18.495766+0000","last_clean":"2023-12-04T05:39:18.495766+0000","last_became_active":"2023-12-04T05:39:17.207977+0000","last_became_peered":"2023-12-04T05:39:17.207977+0000","last_unstale":"2023-12-04T05:39:18.495766+0000","last_undegraded":"2023-12-04T05:39:18.495766+0000","last_fullsized":"2023-12-04T05:39:18.495766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518609+0000","last_change":"2023-12-04T05:39:16.222709+0000","last_active":"2023-12-04T05:39:18.518609+0000","last_peered":"2023-12-04T05:39:18.518609+0000","last_clean":"2023-12-04T05:39:18.518609+0000","last_became_active":"2023-12-04T05:39:16.222155+0000","last_became_peered":"2023-12-04T05:39:16.222155+0000","last_unstale":"2023-12-04T05:39:18.518609+0000","last_undegraded":"2023-12-04T05:39:18.518609+0000","last_fullsized":"2023-12-04T05:39:18.518609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.188}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:39:20.501 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-04T05:39:20.502 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-04T05:39:20.502 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-04T05:39:20.502 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-04T05:39:20.502 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-04T05:39:20.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.635+0000 7f9a12701640 1 Processor -- start 2023-12-04T05:39:20.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.635+0000 7f9a12701640 1 -- start start 2023-12-04T05:39:20.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a12701640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9a0c15b070 0x7f9a0c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a12701640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a0c0cf810 0x7f9a0c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a12701640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0cbaf0 con 0x7f9a0c15b450 2023-12-04T05:39:20.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a12701640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ccb90 con 0x7f9a0c0cf810 2023-12-04T05:39:20.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a12701640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cbdd0 con 0x7f9a0c15b070 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a10c9e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9a0c0cf810 0x7f9a0c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a0b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9a0c15b450 0x7f9a0c0cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58618/0 (socket says 172.21.15.111:58618) 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a0b7fe640 1 -- 172.21.15.111:0/3883324534 learned_addr learned my addr 172.21.15.111:0/3883324534 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.636+0000 7f9a0bfff640 1 --2- 172.21.15.111:0/3883324534 >> v2:172.21.15.192:3300/0 conn(0x7f9a0c15b070 0x7f9a0c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a10c9e640 1 -- 172.21.15.111:0/3883324534 >> v2:172.21.15.192:3300/0 conn(0x7f9a0c15b070 msgr2=0x7f9a0c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a10c9e640 1 --2- 172.21.15.111:0/3883324534 >> v2:172.21.15.192:3300/0 conn(0x7f9a0c15b070 0x7f9a0c0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a10c9e640 1 -- 172.21.15.111:0/3883324534 >> v1:172.21.15.111:6789/0 conn(0x7f9a0c15b450 legacy=0x7f9a0c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a10c9e640 1 -- 172.21.15.111:0/3883324534 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0cc0b0 con 0x7f9a0c0cf810 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a0affd640 1 -- 172.21.15.111:0/3883324534 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567210294 0 0) 0x7f9a0c0cbaf0 con 0x7f9a0c15b450 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a10c9e640 1 --2- 172.21.15.111:0/3883324534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c0cf810 0x7f9a0c0cb4e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c06ef70 tx=0x7f9a0402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af9a9fbee2970f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a0affd640 1 -- 172.21.15.111:0/3883324534 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0402cbf0 con 0x7f9a0c0cf810 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a0affd640 1 -- 172.21.15.111:0/3883324534 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0402cd50 con 0x7f9a0c0cf810 2023-12-04T05:39:20.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.637+0000 7f9a0affd640 1 -- 172.21.15.111:0/3883324534 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04034660 con 0x7f9a0c0cf810 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 -- 172.21.15.111:0/3883324534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c0cf810 msgr2=0x7f9a0c0cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 --2- 172.21.15.111:0/3883324534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c0cf810 0x7f9a0c0cb4e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c06ef70 tx=0x7f9a0402be40 comp rx=0 tx=0).stop 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 -- 172.21.15.111:0/3883324534 shutdown_connections 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 --2- 172.21.15.111:0/3883324534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c0cf810 0x7f9a0c0cb4e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 --2- 172.21.15.111:0/3883324534 >> v2:172.21.15.192:3300/0 conn(0x7f9a0c15b070 0x7f9a0c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 -- 172.21.15.111:0/3883324534 >> 172.21.15.111:0/3883324534 conn(0x7f9a0c071be0 msgr2=0x7f9a0c071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:20.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 -- 172.21.15.111:0/3883324534 shutdown_connections 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.638+0000 7f9a12701640 1 -- 172.21.15.111:0/3883324534 wait complete. 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.639+0000 7f9a12701640 1 Processor -- start 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.639+0000 7f9a12701640 1 -- start start 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a12701640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a0c0cf810 0x7f9a0c153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a12701640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 0x7f9a0c14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a12701640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a0c15b450 0x7f9a0c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a12701640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c152b90 con 0x7f9a0c15b070 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a12701640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c14f3e0 con 0x7f9a0c15b450 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a12701640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c159930 con 0x7f9a0c0cf810 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 0x7f9a0c14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 0x7f9a0c14f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58630/0 (socket says 172.21.15.111:58630) 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 -- 172.21.15.111:0/884140517 learned_addr learned my addr 172.21.15.111:0/884140517 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a10c9e640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a0c15b450 0x7f9a0c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0bfff640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a0c0cf810 0x7f9a0c153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 -- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a0c0cf810 msgr2=0x7f9a0c153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a0c0cf810 0x7f9a0c153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 -- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a0c15b450 msgr2=0x7f9a0c14fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a0c15b450 0x7f9a0c14fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a0b7fe640 1 -- 172.21.15.111:0/884140517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009a70 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.640+0000 7f9a10c9e640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a0c15b450 0x7f9a0c14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.641+0000 7f9a0b7fe640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 0x7f9a0c14f800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c460 tx=0x7f9a0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.641+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000074e0 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.641+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a00007660 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.641+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00004a20 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.641+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c159b80 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.641+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c06dbd0 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.642+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c0693d0 con 0x7f9a0c15b070 2023-12-04T05:39:20.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.642+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9a00004b80 con 0x7f9a0c15b070 2023-12-04T05:39:20.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.643+0000 7f9a08ff9640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f99f806b2a0 0x7f99f806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.643+0000 7f9a0bfff640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f99f806b2a0 0x7f99f806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.643+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a00088de0 con 0x7f9a0c15b070 2023-12-04T05:39:20.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.643+0000 7f9a0bfff640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f99f806b2a0 0x7f99f806d760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99fc006fd0 tx=0x7f99fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.647+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a00053c50 con 0x7f9a0c15b070 2023-12-04T05:39:20.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.857+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9a0c15a180 con 0x7f9a0c15b070 2023-12-04T05:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.860+0000 7f9a08ff9640 1 -- 172.21.15.111:0/884140517 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171577 (secure 0 0 0) 0x7f9a00018070 con 0x7f9a0c15b070 2023-12-04T05:39:20.863 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f99f806b2a0 msgr2=0x7f99f806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f99f806b2a0 0x7f99f806d760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99fc006fd0 tx=0x7f99fc008040 comp rx=0 tx=0).stop 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 msgr2=0x7f9a0c14f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 0x7f9a0c14f800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c460 tx=0x7f9a0000c920 comp rx=0 tx=0).stop 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 shutdown_connections 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f99f806b2a0 0x7f99f806d760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a0c15b450 0x7f9a0c14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a0c15b070 0x7f9a0c14f800 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 --2- 172.21.15.111:0/884140517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a0c0cf810 0x7f9a0c153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 >> 172.21.15.111:0/884140517 conn(0x7f9a0c071be0 msgr2=0x7f9a0c071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 shutdown_connections 2023-12-04T05:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.862+0000 7f9a12701640 1 -- 172.21.15.111:0/884140517 wait complete. 2023-12-04T05:39:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3900317453},{"type":"v1","addr":"172.21.15.111:6833","nonce":3900317453}]},"active_addr":"172.21.15.111:6833/3900317453","active_change":"2023-12-04T05:38:56.464957+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3133010542}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3000926875}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3682226414}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":2412311190}]}]} 2023-12-04T05:39:20.877 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-04T05:39:20.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-04T05:39:20.878 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:20.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.976+0000 7f6f4e51d640 1 Processor -- start 2023-12-04T05:39:20.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.976+0000 7f6f4e51d640 1 -- start start 2023-12-04T05:39:20.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4e51d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 0x7f6f480b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4e51d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480cadf0 con 0x7f6f480b30e0 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4e51d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480ce7c0 con 0x7f6f48146a50 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4e51d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48161ee0 con 0x7f6f48161a80 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4caba640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6f480b30e0 0x7f6f4806be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58626/0 (socket says 172.21.15.111:58626) 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4caba640 1 -- 172.21.15.111:0/3473434495 learned_addr learned my addr 172.21.15.111:0/3473434495 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f477fe640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 0x7f6f480b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942130139 0 0) 0x7f6f480cadf0 con 0x7f6f480b30e0 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f30003520 con 0x7f6f480b30e0 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029334818 0 0) 0x7f6f480ce7c0 con 0x7f6f48146a50 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f480cadf0 con 0x7f6f48146a50 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2859519378 0 0) 0x7f6f30003520 con 0x7f6f480b30e0 2023-12-04T05:39:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 msgr2=0x7f6f480b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 0x7f6f480b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 >> v1:172.21.15.192:6789/0 conn(0x7f6f48146a50 legacy=0x7f6f480cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480b3510 con 0x7f6f480b30e0 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f6f400019f0 con 0x7f6f480b30e0 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 >> v1:172.21.15.111:6789/0 conn(0x7f6f480b30e0 legacy=0x7f6f4806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 0x7f6f30003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f30004d00 0x7f6f30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48161ee0 con 0x7f6f300040e0 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3000bb40 con 0x7f6f30004d00 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3000bd00 con 0x7f6f48161a80 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f477fe640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f47fff640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 0x7f6f30003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.977+0000 7f6f4caba640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f30004d00 0x7f6f30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f477fe640 1 -- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 msgr2=0x7f6f30003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f477fe640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f48161a80 0x7f6f30003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f477fe640 1 -- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f30004d00 msgr2=0x7f6f30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f477fe640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f30004d00 0x7f6f30005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f477fe640 1 -- 172.21.15.111:0/3473434495 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f30003520 con 0x7f6f300040e0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f477fe640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f300047c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009ac0 tx=0x7f6f300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f4a0000e98990d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c02c6b0 con 0x7f6f300040e0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c02c810 con 0x7f6f300040e0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.978+0000 7f6f46ffd640 1 -- 172.21.15.111:0/3473434495 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c02cba0 con 0x7f6f300040e0 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4e51d640 1 -- 172.21.15.111:0/3473434495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 msgr2=0x7f6f300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/3473434495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f300047c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009ac0 tx=0x7f6f300057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4caba640 1 -- 172.21.15.111:0/3473434495 reap_dead start 2023-12-04T05:39:20.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4e51d640 1 -- 172.21.15.111:0/3473434495 shutdown_connections 2023-12-04T05:39:20.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4e51d640 1 -- 172.21.15.111:0/3473434495 >> 172.21.15.111:0/3473434495 conn(0x7f6f4806a4c0 msgr2=0x7f6f4806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:20.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4e51d640 1 -- 172.21.15.111:0/3473434495 shutdown_connections 2023-12-04T05:39:20.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.979+0000 7f6f4e51d640 1 -- 172.21.15.111:0/3473434495 wait complete. 2023-12-04T05:39:20.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.980+0000 7f6f4e51d640 1 Processor -- start 2023-12-04T05:39:20.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.980+0000 7f6f4e51d640 1 -- start start 2023-12-04T05:39:20.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4e51d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f48159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4e51d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f481f3360 0x7f6f4815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4e51d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f481417e0 0x7f6f4815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4e51d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48141fa0 con 0x7f6f300040e0 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4e51d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b52c0 con 0x7f6f481417e0 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4e51d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b55a0 con 0x7f6f481f3360 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f48159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f48159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58646/0 (socket says 172.21.15.111:58646) 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 -- 172.21.15.111:0/2971554088 learned_addr learned my addr 172.21.15.111:0/2971554088 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 -- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f481f3360 msgr2=0x7f6f4815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f481f3360 0x7f6f4815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f4caba640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f481417e0 0x7f6f4815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 -- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f481417e0 msgr2=0x7f6f4815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f481417e0 0x7f6f4815ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:20.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 -- 172.21.15.111:0/2971554088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480b5880 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.981+0000 7f6f47fff640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f48159cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800f810 tx=0x7f6f3800fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.982+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380014e0 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.982+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f380052e0 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.982+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480b5b60 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.982+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800d950 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.982+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4815d460 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.983+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f480b5da0 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.983+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6f3800dab0 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.983+0000 7f6f44ff9640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3406b2d0 0x7f6f3406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.984+0000 7f6f477fe640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3406b2d0 0x7f6f3406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.984+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f38089c40 con 0x7f6f300040e0 2023-12-04T05:39:20.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.984+0000 7f6f477fe640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3406b2d0 0x7f6f3406d790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c036000 tx=0x7f6f3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:20.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:20.988+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f480b5da0 con 0x7f6f300040e0 2023-12-04T05:39:21.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.153+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f480b5da0 con 0x7f6f300040e0 2023-12-04T05:39:21.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.154+0000 7f6f44ff9640 1 -- 172.21.15.111:0/2971554088 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f6f480b5da0 con 0x7f6f300040e0 2023-12-04T05:39:21.156 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:21.156 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:21.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3406b2d0 msgr2=0x7f6f3406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3406b2d0 0x7f6f3406d790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c036000 tx=0x7f6f3c02e000 comp rx=0 tx=0).stop 2023-12-04T05:39:21.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 msgr2=0x7f6f48159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f48159cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800f810 tx=0x7f6f3800fcd0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 shutdown_connections 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3406b2d0 0x7f6f3406d790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f481417e0 0x7f6f4815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f481f3360 0x7f6f4815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 --2- 172.21.15.111:0/2971554088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f300040e0 0x7f6f48159cf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 >> 172.21.15.111:0/2971554088 conn(0x7f6f4806a4c0 msgr2=0x7f6f4806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 shutdown_connections 2023-12-04T05:39:21.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.156+0000 7f6f4e51d640 1 -- 172.21.15.111:0/2971554088 wait complete. 2023-12-04T05:39:21.169 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-04T05:39:21.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:21.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.269+0000 7f5aba9c9640 1 Processor -- start 2023-12-04T05:39:21.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.269+0000 7f5aba9c9640 1 -- start start 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5aba9c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab406bc90 0x7f5ab406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5aba9c9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5ab40b3590 0x7f5ab40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5aba9c9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab40cf230 con 0x7f5ab415d6c0 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5aba9c9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab406bb10 con 0x7f5ab40b3590 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5aba9c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40cf4d0 con 0x7f5ab406bc90 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5ab8f66640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5ab40b3590 0x7f5ab40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5ab37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5ab415d6c0 0x7f5ab406c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58638/0 (socket says 172.21.15.111:58638) 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.270+0000 7f5ab37fe640 1 -- 172.21.15.111:0/3563214733 learned_addr learned my addr 172.21.15.111:0/3563214733 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab3fff640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab406bc90 0x7f5ab406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab8f66640 1 -- 172.21.15.111:0/3563214733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab406bc90 msgr2=0x7f5ab406b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab8f66640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab406bc90 0x7f5ab406b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab8f66640 1 -- 172.21.15.111:0/3563214733 >> v1:172.21.15.111:6789/0 conn(0x7f5ab415d6c0 legacy=0x7f5ab406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab8f66640 1 -- 172.21.15.111:0/3563214733 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40cf7b0 con 0x7f5ab40b3590 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab2ffd640 1 -- 172.21.15.111:0/3563214733 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567749271 0 0) 0x7f5ab40cf230 con 0x7f5ab415d6c0 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab3fff640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab406bc90 0x7f5ab406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.271+0000 7f5ab8f66640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab40cb5b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009d90 tx=0x7f5aac02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7319a35c64034c8c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5ab2ffd640 1 -- 172.21.15.111:0/3563214733 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac02cbc0 con 0x7f5ab40b3590 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5ab2ffd640 1 -- 172.21.15.111:0/3563214733 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac02cd20 con 0x7f5ab40b3590 2023-12-04T05:39:21.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5ab2ffd640 1 -- 172.21.15.111:0/3563214733 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac034660 con 0x7f5ab40b3590 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 -- 172.21.15.111:0/3563214733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 msgr2=0x7f5ab40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab40cb5b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009d90 tx=0x7f5aac02be10 comp rx=0 tx=0).stop 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 -- 172.21.15.111:0/3563214733 shutdown_connections 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab40cb5b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/3563214733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab406bc90 0x7f5ab406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 -- 172.21.15.111:0/3563214733 >> 172.21.15.111:0/3563214733 conn(0x7f5ab40b2a70 msgr2=0x7f5ab40b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 -- 172.21.15.111:0/3563214733 shutdown_connections 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.272+0000 7f5aba9c9640 1 -- 172.21.15.111:0/3563214733 wait complete. 2023-12-04T05:39:21.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.273+0000 7f5aba9c9640 1 Processor -- start 2023-12-04T05:39:21.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.273+0000 7f5aba9c9640 1 -- start start 2023-12-04T05:39:21.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5aba9c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab406bc90 0x7f5ab40af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5aba9c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5aba9c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4155470 0x7f5ab415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5aba9c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40af730 con 0x7f5ab40b3590 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5aba9c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab414b8c0 con 0x7f5ab406bc90 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5aba9c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab415b8c0 con 0x7f5ab4155470 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab414b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58654/0 (socket says 172.21.15.111:58654) 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5ab37fe640 1 -- 172.21.15.111:0/764789813 learned_addr learned my addr 172.21.15.111:0/764789813 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5ab3fff640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab406bc90 0x7f5ab40af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.274+0000 7f5ab8f66640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4155470 0x7f5ab415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab37fe640 1 -- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4155470 msgr2=0x7f5ab415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab37fe640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4155470 0x7f5ab415c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab37fe640 1 -- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab406bc90 msgr2=0x7f5ab40af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab37fe640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab406bc90 0x7f5ab40af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab37fe640 1 -- 172.21.15.111:0/764789813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac009a70 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab37fe640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab414b380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800c430 tx=0x7f5aa800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80074e0 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa8007660 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab415bba0 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004a20 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.275+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab415c160 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.276+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab4067460 con 0x7f5ab40b3590 2023-12-04T05:39:21.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.276+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5aa8004b80 con 0x7f5ab40b3590 2023-12-04T05:39:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.277+0000 7f5ab0ff9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5aa006b2a0 0x7f5aa006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.277+0000 7f5ab3fff640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5aa006b2a0 0x7f5aa006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.277+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5aa8089a00 con 0x7f5ab40b3590 2023-12-04T05:39:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.277+0000 7f5ab3fff640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5aa006b2a0 0x7f5aa006d760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4072bf0 tx=0x7f5aa4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.281+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5aa8054870 con 0x7f5ab40b3590 2023-12-04T05:39:21.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.449+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ab40b24c0 con 0x7f5ab40b3590 2023-12-04T05:39:21.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.450+0000 7f5ab0ff9640 1 -- 172.21.15.111:0/764789813 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f5aa8057e90 con 0x7f5ab40b3590 2023-12-04T05:39:21.451 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:21.451 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:21.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5aa006b2a0 msgr2=0x7f5aa006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5aa006b2a0 0x7f5aa006d760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4072bf0 tx=0x7f5aa4008040 comp rx=0 tx=0).stop 2023-12-04T05:39:21.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 msgr2=0x7f5ab414b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab414b380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800c430 tx=0x7f5aa800c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 shutdown_connections 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5aa006b2a0 0x7f5aa006d760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4155470 0x7f5ab415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.451+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ab40b3590 0x7f5ab414b380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.452+0000 7f5aba9c9640 1 --2- 172.21.15.111:0/764789813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab406bc90 0x7f5ab40af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.452+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 >> 172.21.15.111:0/764789813 conn(0x7f5ab40b2a70 msgr2=0x7f5ab40b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.452+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 shutdown_connections 2023-12-04T05:39:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.452+0000 7f5aba9c9640 1 -- 172.21.15.111:0/764789813 wait complete. 2023-12-04T05:39:21.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-04T05:39:21.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-04T05:39:21.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-04T05:39:21.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-04T05:39:21.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-04T05:39:21.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-04T05:39:21.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-04T05:39:21.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-04T05:39:21.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.636+0000 7f72b794c640 1 Processor -- start 2023-12-04T05:39:21.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.637+0000 7f72b794c640 1 -- start start 2023-12-04T05:39:21.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.637+0000 7f8153631640 1 Processor -- start 2023-12-04T05:39:21.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.637+0000 7f8153631640 1 -- start start 2023-12-04T05:39:21.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7fc35c82c640 1 Processor -- start 2023-12-04T05:39:21.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7fc35c82c640 1 -- start start 2023-12-04T05:39:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f990e316640 1 Processor -- start 2023-12-04T05:39:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f990e316640 1 -- start start 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f72b794c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f72b016e5f0 0x7f72b00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f72b794c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b008a700 con 0x7f72b0071900 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f72b794c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b016ea50 con 0x7f72b016db00 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f72b794c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00b3990 con 0x7f72b016e5f0 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f72b5ee9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f72b0071900 0x7f72b0071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58652/0 (socket says 172.21.15.111:58652) 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f72b5ee9640 1 -- 172.21.15.111:0/3913706519 learned_addr learned my addr 172.21.15.111:0/3913706519 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f72b4ee7640 1 --2- 172.21.15.111:0/3913706519 >> v2:172.21.15.192:3300/0 conn(0x7f72b016e5f0 0x7f72b00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f8153631640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f814c0b3790 0x7f814c1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f8153631640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f814c0d09c0 0x7f814c1653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f8153631640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c0b4220 con 0x7f814c0d05e0 2023-12-04T05:39:21.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f8153631640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c165fb0 con 0x7f814c0b3790 2023-12-04T05:39:21.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.638+0000 7f8153631640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c166290 con 0x7f814c0d09c0 2023-12-04T05:39:21.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f66c157a640 1 Processor -- start 2023-12-04T05:39:21.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f66c157a640 1 -- start start 2023-12-04T05:39:21.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f81513cd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f814c0d05e0 0x7f814c0b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43032/0 (socket says 172.21.15.111:43032) 2023-12-04T05:39:21.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f81513cd640 1 -- 172.21.15.111:0/3187415052 learned_addr learned my addr 172.21.15.111:0/3187415052 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f8150bcc640 1 --2- 172.21.15.111:0/3187415052 >> v2:172.21.15.111:3300/0 conn(0x7f814c0b3790 0x7f814c1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f8151bce640 1 --2- 172.21.15.111:0/3187415052 >> v2:172.21.15.192:3300/0 conn(0x7f814c0d09c0 0x7f814c1653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217205777 0 0) 0x7f72b008a700 con 0x7f72b0071900 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc35c82c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc3500eacd0 0x7fc35004fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc35c82c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35004ffa0 0x7fc350037ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc35c82c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc350054300 con 0x7fc3500380e0 2023-12-04T05:39:21.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc35c82c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3500536f0 con 0x7fc3500eacd0 2023-12-04T05:39:21.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc35c82c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500539d0 con 0x7fc35004ffa0 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990e316640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9908161680 0x7f990806e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990e316640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f99080b4f50 0x7f99080b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990e316640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990806b390 con 0x7f990806aed0 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990e316640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990806d270 con 0x7f9908161680 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990e316640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990806d550 con 0x7f99080b4f50 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990c8b3640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f990806aed0 0x7f99080acd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43048/0 (socket says 172.21.15.111:43048) 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f990c8b3640 1 -- 172.21.15.111:0/452281417 learned_addr learned my addr 172.21.15.111:0/452281417 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f9907fff640 1 --2- 172.21.15.111:0/452281417 >> v2:172.21.15.111:3300/0 conn(0x7f9908161680 0x7f990806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f99077fe640 1 --2- 172.21.15.111:0/452281417 >> v2:172.21.15.192:3300/0 conn(0x7f99080b4f50 0x7f99080b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.639+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f729c003520 con 0x7f72b0071900 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f9906ffd640 1 -- 172.21.15.111:0/452281417 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190054660 0 0) 0x7f990806b390 con 0x7f990806aed0 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f9906ffd640 1 -- 172.21.15.111:0/452281417 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f8003520 con 0x7f990806aed0 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f99077fe640 1 -- 172.21.15.111:0/452281417 >> v2:172.21.15.111:3300/0 conn(0x7f9908161680 msgr2=0x7f990806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f99077fe640 1 --2- 172.21.15.111:0/452281417 >> v2:172.21.15.111:3300/0 conn(0x7f9908161680 0x7f990806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f99077fe640 1 -- 172.21.15.111:0/452281417 >> v1:172.21.15.192:6789/0 conn(0x7f990806aed0 legacy=0x7f99080acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f99077fe640 1 -- 172.21.15.111:0/452281417 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990806d830 con 0x7f99080b4f50 2023-12-04T05:39:21.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650050416 0 0) 0x7f72b016ea50 con 0x7f72b016db00 2023-12-04T05:39:21.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f9a4b59e640 1 Processor -- start 2023-12-04T05:39:21.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f66c157a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f66bc06c950 0x7f66bc06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b008a700 con 0x7f72b016db00 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603963715 0 0) 0x7f729c003520 con 0x7f72b0071900 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 >> v2:172.21.15.192:3300/0 conn(0x7f72b016e5f0 msgr2=0x7f72b00713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 --2- 172.21.15.111:0/3913706519 >> v2:172.21.15.192:3300/0 conn(0x7f72b016e5f0 0x7f72b00713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 >> v1:172.21.15.192:6789/0 conn(0x7f72b016db00 legacy=0x7f72b016dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f66c157a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0cc800 0x7f66bc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f66c157a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66bc0cea30 con 0x7f66bc153660 2023-12-04T05:39:21.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f66c157a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc0b2850 con 0x7f66bc06c950 2023-12-04T05:39:21.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00b3c70 con 0x7f72b0071900 2023-12-04T05:39:21.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f66c157a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc06d340 con 0x7f66bc0cc800 2023-12-04T05:39:21.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f72a8005200 con 0x7f72b0071900 2023-12-04T05:39:21.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f66bb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f66bc06c950 0x7f66bc06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 >> v1:172.21.15.111:6789/0 conn(0x7f72b0071900 legacy=0x7f72b0071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f729c003b00 0x7f729c003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c0047c0 0x7f729c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f729c005280 0x7f729c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00b3990 con 0x7f729c0047c0 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0149e0 con 0x7f729c005280 2023-12-04T05:39:21.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c014ba0 con 0x7f729c003b00 2023-12-04T05:39:21.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f729c005280 0x7f729c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72b56e8640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f729c003b00 0x7f729c003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f72b4ee7640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c0047c0 0x7f729c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66bb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f66bc06c950 0x7f66bc06ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58676/0 (socket says 172.21.15.111:58676) 2023-12-04T05:39:21.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66bb7fe640 1 -- 172.21.15.111:0/673528459 learned_addr learned my addr 172.21.15.111:0/673528459 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66c0d79640 1 --2- 172.21.15.111:0/673528459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0cc800 0x7f66bc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f84e72a8640 1 Processor -- start 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f84e72a8640 1 -- start start 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f84e72a8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84d800c070 0x7f84d8038050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f84e72a8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84d8053c40 0x7f84d80df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f84e72a8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d8054490 con 0x7f84d8033ec0 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f84e72a8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d80541b0 con 0x7f84d8053c40 2023-12-04T05:39:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f84e72a8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8005f00 con 0x7f84d800c070 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8143fff640 1 -- 172.21.15.111:0/3187415052 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499643479 0 0) 0x7f814c0b4220 con 0x7f814c0d05e0 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8143fff640 1 -- 172.21.15.111:0/3187415052 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8138003520 con 0x7f814c0d05e0 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8150bcc640 1 -- 172.21.15.111:0/3187415052 >> v2:172.21.15.192:3300/0 conn(0x7f814c0d09c0 msgr2=0x7f814c1653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8143fff640 1 -- 172.21.15.111:0/3187415052 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2183735865 0 0) 0x7f8138003520 con 0x7f814c0d05e0 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8150bcc640 1 --2- 172.21.15.111:0/3187415052 >> v2:172.21.15.192:3300/0 conn(0x7f814c0d09c0 0x7f814c1653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8150bcc640 1 -- 172.21.15.111:0/3187415052 >> v1:172.21.15.192:6789/0 conn(0x7f814c0d05e0 legacy=0x7f814c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f8150bcc640 1 -- 172.21.15.111:0/3187415052 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c161ef0 con 0x7f814c0b3790 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f84e5aa5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84d8053c40 0x7f84d80df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f84e5aa5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84d8053c40 0x7f84d80df160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57034/0 (socket says 172.21.15.111:57034) 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f84e5aa5640 1 -- 172.21.15.111:0/2770905739 learned_addr learned my addr 172.21.15.111:0/2770905739 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc356d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc3500eacd0 0x7fc35004fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc356d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc3500eacd0 0x7fc35004fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58674/0 (socket says 172.21.15.111:58674) 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc356d9d640 1 -- 172.21.15.111:0/2722631440 learned_addr learned my addr 172.21.15.111:0/2722631440 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc35659c640 1 --2- 172.21.15.111:0/2722631440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35004ffa0 0x7fc350037ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc356d9d640 1 -- 172.21.15.111:0/2722631440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35004ffa0 msgr2=0x7fc350037ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7fc356d9d640 1 --2- 172.21.15.111:0/2722631440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35004ffa0 0x7fc350037ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7fc355d9b640 1 -- 172.21.15.111:0/2722631440 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746317518 0 0) 0x7fc350054300 con 0x7fc3500380e0 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7fc356d9d640 1 -- 172.21.15.111:0/2722631440 >> v1:172.21.15.111:6789/0 conn(0x7fc3500380e0 legacy=0x7fc350038570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7fc356d9d640 1 -- 172.21.15.111:0/2722631440 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350053cb0 con 0x7fc3500eacd0 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f72b5ee9640 1 -- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f729c003b00 msgr2=0x7f729c003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f729c003b00 0x7f729c003ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f72b5ee9640 1 -- 172.21.15.111:0/3913706519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c0047c0 msgr2=0x7f729c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c0047c0 0x7f729c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f72b5ee9640 1 -- 172.21.15.111:0/3913706519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c003520 con 0x7f729c005280 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f72b5ee9640 1 -- 172.21.15.111:0/3913706519 reap_dead start 2023-12-04T05:39:21.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f729c005280 0x7f729c00b980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72a8008fd0 tx=0x7f729c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=385c2d33f5c76a24 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f99077fe640 1 --2- 172.21.15.111:0/452281417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 0x7f99080b2c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9908070e00 tx=0x7f98fc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1355d6d88691d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9906ffd640 1 -- 172.21.15.111:0/452281417 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc031830 con 0x7f99080b4f50 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9906ffd640 1 -- 172.21.15.111:0/452281417 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc031990 con 0x7f99080b4f50 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9906ffd640 1 -- 172.21.15.111:0/452281417 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc031d20 con 0x7f99080b4f50 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.640+0000 7f9a4b59e640 1 -- start start 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a4b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 0x7f9a4c0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a4b59e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0ce100 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a4b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0b38f0 con 0x7f9a4c0cd440 2023-12-04T05:39:21.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a4b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0abd30 con 0x7f9a4c071940 2023-12-04T05:39:21.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f8151bce640 1 --2- 172.21.15.111:0/3187415052 >> v2:172.21.15.192:3300/0 conn(0x7f814c0d09c0 0x7f814c1653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:21.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66bb7fe640 1 -- 172.21.15.111:0/673528459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0cc800 msgr2=0x7f66bc0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a49d9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 0x7f9a4c0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a49d9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 0x7f9a4c0b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:39996/0 (socket says 172.21.15.111:39996) 2023-12-04T05:39:21.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1355800760 learned_addr learned my addr 172.21.15.111:0/1355800760 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66bb7fe640 1 --2- 172.21.15.111:0/673528459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0cc800 0x7f66bc0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66bb7fe640 1 -- 172.21.15.111:0/673528459 >> v1:172.21.15.192:6789/0 conn(0x7f66bc153660 legacy=0x7f66bc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.641+0000 7f66bb7fe640 1 -- 172.21.15.111:0/673528459 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0b1eb0 con 0x7f66bc06c950 2023-12-04T05:39:21.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f8150bcc640 1 --2- 172.21.15.111:0/3187415052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c0b3790 0x7f814c1629d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f813c0097a0 tx=0x7f813c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111f80e2920f7c30 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8008270 con 0x7f729c005280 2023-12-04T05:39:21.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a802cb10 con 0x7f729c005280 2023-12-04T05:39:21.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f72a7fff640 1 -- 172.21.15.111:0/3913706519 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80064e0 con 0x7f729c005280 2023-12-04T05:39:21.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1355800760 >> v1:172.21.15.192:6789/0 conn(0x7f9a4c0cd440 legacy=0x7f9a4c071230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:39:21.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f8143fff640 1 -- 172.21.15.111:0/3187415052 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148001670 con 0x7f814c0b3790 2023-12-04T05:39:21.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f8143fff640 1 -- 172.21.15.111:0/3187415052 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c02e050 con 0x7f814c0b3790 2023-12-04T05:39:21.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f8143fff640 1 -- 172.21.15.111:0/3187415052 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c030710 con 0x7f814c0b3790 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 -- 172.21.15.111:0/452281417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 msgr2=0x7f99080b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743252905 0 0) 0x7f84d8054490 con 0x7f84d8033ec0 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84cc003520 con 0x7f84d8033ec0 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952524738 0 0) 0x7f84cc003520 con 0x7f84d8033ec0 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84d800c070 msgr2=0x7f84d8038050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84d800c070 0x7f84d8038050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 >> v2:172.21.15.192:3300/0 conn(0x7f84d8053c40 msgr2=0x7f84d80df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 --2- 172.21.15.111:0/2770905739 >> v2:172.21.15.192:3300/0 conn(0x7f84d8053c40 0x7f84d80df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80061e0 con 0x7f84d8033ec0 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f84e004eda0 con 0x7f84d8033ec0 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f66bb7fe640 1 --2- 172.21.15.111:0/673528459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc06c950 0x7f66bc06ce00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009bd0 tx=0x7f66ac02d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f417853817fbf6b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 >> v1:172.21.15.192:6789/0 conn(0x7f84d8033ec0 legacy=0x7f84d804fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84cc003970 0x7f84cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004310 0x7f84cc0047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84cc004f90 0x7f84cc007960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 --2- 172.21.15.111:0/452281417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 0x7f99080b2c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9908070e00 tx=0x7f98fc02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80541b0 con 0x7f84cc004f90 2023-12-04T05:39:21.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8005f00 con 0x7f84cc003970 2023-12-04T05:39:21.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc007f20 con 0x7f84cc004310 2023-12-04T05:39:21.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84cc004f90 0x7f84cc007960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 -- 172.21.15.111:0/452281417 shutdown_connections 2023-12-04T05:39:21.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 --2- 172.21.15.111:0/452281417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 0x7f99080b2c60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 --2- 172.21.15.111:0/452281417 >> v2:172.21.15.111:3300/0 conn(0x7f9908161680 0x7f990806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 -- 172.21.15.111:0/452281417 >> 172.21.15.111:0/452281417 conn(0x7f99080b1960 msgr2=0x7f99080b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e62a6640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84cc003970 0x7f84cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f72b794c640 1 -- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f729c005280 msgr2=0x7f729c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f72b794c640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f729c005280 0x7f729c00b980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72a8008fd0 tx=0x7f729c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7fc356d9d640 1 --2- 172.21.15.111:0/2722631440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3500eacd0 0x7fc35004fa60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009870 tx=0x7fc34c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d69969c489967a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f66baffd640 1 -- 172.21.15.111:0/673528459 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0353f0 con 0x7f66bc06c950 2023-12-04T05:39:21.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1355800760 >> v1:172.21.15.111:6789/0 conn(0x7f9a4c0b34b0 legacy=0x7f9a4c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.642+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1355800760 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c0cddd0 con 0x7f9a4c071940 2023-12-04T05:39:21.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e6aa7640 1 -- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004310 msgr2=0x7f84cc0047e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-04T05:39:21.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004310 0x7f84cc0047e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e6aa7640 1 -- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84cc003970 msgr2=0x7f84cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f66baffd640 1 -- 172.21.15.111:0/673528459 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac035550 con 0x7f66bc06c950 2023-12-04T05:39:21.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f66baffd640 1 -- 172.21.15.111:0/673528459 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0358e0 con 0x7f66bc06c950 2023-12-04T05:39:21.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f72b794c640 1 -- 172.21.15.111:0/3913706519 shutdown_connections 2023-12-04T05:39:21.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f72b794c640 1 --2- 172.21.15.111:0/3913706519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f729c005280 0x7f729c00b980 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f72b794c640 1 -- 172.21.15.111:0/3913706519 >> 172.21.15.111:0/3913706519 conn(0x7f72b006c250 msgr2=0x7f72b006c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7fc355d9b640 1 -- 172.21.15.111:0/2722631440 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c030840 con 0x7fc3500eacd0 2023-12-04T05:39:21.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f9a49d9b640 1 --2- 172.21.15.111:0/1355800760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 0x7f9a4c0b2f70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009e80 tx=0x7f9a4402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea4797825b3d55ac server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7fc355d9b640 1 -- 172.21.15.111:0/2722631440 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c0309a0 con 0x7fc3500eacd0 2023-12-04T05:39:21.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7fc355d9b640 1 -- 172.21.15.111:0/2722631440 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c030d30 con 0x7fc3500eacd0 2023-12-04T05:39:21.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84cc003970 0x7f84cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f84e6aa7640 1 -- 172.21.15.111:0/2770905739 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc003520 con 0x7f84cc004f90 2023-12-04T05:39:21.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f84e62a6640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84cc003970 0x7f84cc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:21.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 -- 172.21.15.111:0/3187415052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c0b3790 msgr2=0x7f814c1629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 --2- 172.21.15.111:0/3187415052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c0b3790 0x7f814c1629d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f813c0097a0 tx=0x7f813c02b7b0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 -- 172.21.15.111:0/3187415052 shutdown_connections 2023-12-04T05:39:21.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 --2- 172.21.15.111:0/3187415052 >> v2:172.21.15.192:3300/0 conn(0x7f814c0d09c0 0x7f814c1653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 --2- 172.21.15.111:0/3187415052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c0b3790 0x7f814c1629d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 -- 172.21.15.111:0/3187415052 >> 172.21.15.111:0/3187415052 conn(0x7f814c06a8b0 msgr2=0x7f814c06acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f72b794c640 1 -- 172.21.15.111:0/3913706519 shutdown_connections 2023-12-04T05:39:21.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f9a4959a640 1 -- 172.21.15.111:0/1355800760 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4402ca50 con 0x7f9a4c071940 2023-12-04T05:39:21.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f72b794c640 1 -- 172.21.15.111:0/3913706519 wait complete. 2023-12-04T05:39:21.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f9a4959a640 1 -- 172.21.15.111:0/1355800760 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4402cbb0 con 0x7f9a4c071940 2023-12-04T05:39:21.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.643+0000 7f990e316640 1 -- 172.21.15.111:0/452281417 shutdown_connections 2023-12-04T05:39:21.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 -- 172.21.15.111:0/3187415052 shutdown_connections 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f990e316640 1 -- 172.21.15.111:0/452281417 wait complete. 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f8153631640 1 -- 172.21.15.111:0/3187415052 wait complete. 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 -- 172.21.15.111:0/673528459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc06c950 msgr2=0x7f66bc06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 --2- 172.21.15.111:0/673528459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc06c950 0x7f66bc06ce00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009bd0 tx=0x7f66ac02d340 comp rx=0 tx=0).stop 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 -- 172.21.15.111:0/673528459 shutdown_connections 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 --2- 172.21.15.111:0/673528459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0cc800 0x7f66bc0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 --2- 172.21.15.111:0/673528459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc06c950 0x7f66bc06ce00 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009bd0 tx=0x7f66ac02d340 comp rx=0 tx=0).stop 2023-12-04T05:39:21.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 -- 172.21.15.111:0/673528459 >> 172.21.15.111:0/673528459 conn(0x7f66bc06a8b0 msgr2=0x7f66bc06acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 -- 172.21.15.111:0/2722631440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3500eacd0 msgr2=0x7fc35004fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 --2- 172.21.15.111:0/2722631440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3500eacd0 0x7fc35004fa60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009870 tx=0x7fc34c02b980 comp rx=0 tx=0).stop 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 -- 172.21.15.111:0/2722631440 shutdown_connections 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 --2- 172.21.15.111:0/2722631440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35004ffa0 0x7fc350037ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 --2- 172.21.15.111:0/2722631440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3500eacd0 0x7fc35004fa60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 -- 172.21.15.111:0/2722631440 >> 172.21.15.111:0/2722631440 conn(0x7fc350009f80 msgr2=0x7fc35000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 -- 172.21.15.111:0/2722631440 shutdown_connections 2023-12-04T05:39:21.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7fc35c82c640 1 -- 172.21.15.111:0/2722631440 wait complete. 2023-12-04T05:39:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84cc004f90 0x7f84cc007960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f84e0063ea0 tx=0x7f84cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640a23ce1b427268 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1355800760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 msgr2=0x7f9a4c0b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f9a4b59e640 1 --2- 172.21.15.111:0/1355800760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 0x7f9a4c0b2f70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009e80 tx=0x7f9a4402beb0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 -- 172.21.15.111:0/673528459 shutdown_connections 2023-12-04T05:39:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00ad600 con 0x7f84cc004f90 2023-12-04T05:39:21.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.644+0000 7f452f3ce640 1 Processor -- start 2023-12-04T05:39:21.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f452f3ce640 1 -- start start 2023-12-04T05:39:21.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e00a9050 con 0x7f84cc004f90 2023-12-04T05:39:21.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1355800760 shutdown_connections 2023-12-04T05:39:21.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f9a4b59e640 1 --2- 172.21.15.111:0/1355800760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c071940 0x7f9a4c0b2f70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1355800760 >> 172.21.15.111:0/1355800760 conn(0x7f9a4c0b0950 msgr2=0x7f9a4c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f84e52a4640 1 -- 172.21.15.111:0/2770905739 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00ada40 con 0x7f84cc004f90 2023-12-04T05:39:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1355800760 shutdown_connections 2023-12-04T05:39:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452f3ce640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f452806bf20 0x7f452806de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f990e316640 1 Processor -- start 2023-12-04T05:39:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452f3ce640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4528071350 con 0x7f45280cdcd0 2023-12-04T05:39:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452d96b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f45280cdcd0 0x7f45280ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58662/0 (socket says 172.21.15.111:58662) 2023-12-04T05:39:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452d96b640 1 -- 172.21.15.111:0/1064648371 learned_addr learned my addr 172.21.15.111:0/1064648371 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f84e72a8640 1 -- 172.21.15.111:0/2770905739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84cc004f90 msgr2=0x7f84cc007960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452c969640 1 --2- 172.21.15.111:0/1064648371 >> v2:172.21.15.192:3300/0 conn(0x7f452806bf20 0x7f452806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4528071630 con 0x7f452806ee20 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452806f200 con 0x7f452806bf20 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f84e72a8640 1 --2- 172.21.15.111:0/2770905739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84cc004f90 0x7f84cc007960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f84e0063ea0 tx=0x7f84cc0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f84e6aa7640 1 -- 172.21.15.111:0/2770905739 reap_dead start 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f84e72a8640 1 -- 172.21.15.111:0/2770905739 shutdown_connections 2023-12-04T05:39:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f84e72a8640 1 -- 172.21.15.111:0/2770905739 >> 172.21.15.111:0/2770905739 conn(0x7f84d8009f80 msgr2=0x7f84d800a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f8153631640 1 Processor -- start 2023-12-04T05:39:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f72b794c640 1 Processor -- start 2023-12-04T05:39:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7fc35c82c640 1 Processor -- start 2023-12-04T05:39:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f72b794c640 1 -- start start 2023-12-04T05:39:21.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7fc35c82c640 1 -- start start 2023-12-04T05:39:21.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f451ffff640 1 -- 172.21.15.111:0/1064648371 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208127112 0 0) 0x7f4528071350 con 0x7f45280cdcd0 2023-12-04T05:39:21.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f451ffff640 1 -- 172.21.15.111:0/1064648371 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4514003520 con 0x7f45280cdcd0 2023-12-04T05:39:21.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f452c969640 1 -- 172.21.15.111:0/1064648371 >> v1:172.21.15.192:6789/0 conn(0x7f452806ee20 legacy=0x7f452806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f452c969640 1 -- 172.21.15.111:0/1064648371 >> v1:172.21.15.111:6789/0 conn(0x7f45280cdcd0 legacy=0x7f45280ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.646+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1355800760 wait complete. 2023-12-04T05:39:21.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f9a4b59e640 1 Processor -- start 2023-12-04T05:39:21.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f9a4b59e640 1 -- start start 2023-12-04T05:39:21.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f8153631640 1 -- start start 2023-12-04T05:39:21.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f990e316640 1 -- start start 2023-12-04T05:39:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990e316640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f990806aed0 0x7f9908150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990e316640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 0x7f99081519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990e316640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9908161680 0x7f99080cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35c82c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3500380e0 0x7fc3501809d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35c82c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc35004ffa0 0x7fc3500dce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35c82c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3500eacd0 0x7fc350049700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990e316640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080cdbf0 con 0x7f9908161680 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990e316640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080cded0 con 0x7f99080b4f50 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990e316640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99080ce1b0 con 0x7f990806aed0 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35c82c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350180f10 con 0x7fc35004ffa0 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8153631640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f814c0d05e0 0x7f814c1ed860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8153631640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f814c0d09c0 0x7f814c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8153631640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c1eca50 0x7f814c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35c82c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350043650 con 0x7fc3500eacd0 2023-12-04T05:39:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f452c969640 1 -- 172.21.15.111:0/1064648371 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528071910 con 0x7f452806bf20 2023-12-04T05:39:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b794c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c005280 0x7f72b016d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b794c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b0203980 0x7f72b015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b794c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72b015a860 0x7f72b015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8153631640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c14a5e0 con 0x7f814c1eca50 2023-12-04T05:39:21.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8153631640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c1edda0 con 0x7f814c0d05e0 2023-12-04T05:39:21.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8153631640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c1ec5c0 con 0x7f814c0d09c0 2023-12-04T05:39:21.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35c82c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc350043930 con 0x7fc3500380e0 2023-12-04T05:39:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b794c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015cf10 con 0x7f729c005280 2023-12-04T05:39:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b794c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0203e00 con 0x7f72b0203980 2023-12-04T05:39:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b794c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015d100 con 0x7f72b015a860 2023-12-04T05:39:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990c8b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9908161680 0x7f99080cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc35004ffa0 0x7fc3500dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc356d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3500380e0 0x7fc3501809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990c8b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9908161680 0x7f99080cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58708/0 (socket says 172.21.15.111:58708) 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc356d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3500380e0 0x7fc3501809d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40036/0 (socket says 172.21.15.111:40036) 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc356d9d640 1 -- 172.21.15.111:0/3689049521 learned_addr learned my addr 172.21.15.111:0/3689049521 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc35004ffa0 0x7fc3500dce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58692/0 (socket says 172.21.15.111:58692) 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7fc35759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3500eacd0 0x7fc350049700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8151bce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c1eca50 0x7f814c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8151bce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c1eca50 0x7f814c15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58728/0 (socket says 172.21.15.111:58728) 2023-12-04T05:39:21.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f8151bce640 1 -- 172.21.15.111:0/2170322232 learned_addr learned my addr 172.21.15.111:0/2170322232 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8150bcc640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f814c0d09c0 0x7f814c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f990c8b3640 1 -- 172.21.15.111:0/508173389 learned_addr learned my addr 172.21.15.111:0/508173389 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f81513cd640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f814c0d05e0 0x7f814c1ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f452c969640 1 --2- 172.21.15.111:0/1064648371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806bf20 0x7f452806de20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45180097a0 tx=0x7f451802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feefdf317090b072 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.645+0000 7f66c157a640 1 -- 172.21.15.111:0/673528459 wait complete. 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f66c157a640 1 Processor -- start 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.647+0000 7f66c157a640 1 -- start start 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f66c157a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc0cc800 0x7f66bc157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66c157a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc153660 0x7f66bc155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66c157a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc1feec0 0x7f66bc070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66bb7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc153660 0x7f66bc155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66bb7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc153660 0x7f66bc155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40064/0 (socket says 172.21.15.111:40064) 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66bb7fe640 1 -- 172.21.15.111:0/1105124951 learned_addr learned my addr 172.21.15.111:0/1105124951 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0c3d80 con 0x7f66bc1feec0 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc156700 con 0x7f66bc0cc800 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc1569e0 con 0x7f66bc153660 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a4c071940 0x7f9a4c14b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b56e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c005280 0x7f72b016d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b5ee9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72b015a860 0x7f72b015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f990c8b3640 1 -- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f990806aed0 msgr2=0x7f9908150050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-04T05:39:21.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b5ee9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72b015a860 0x7f72b015fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40052/0 (socket says 172.21.15.111:40052) 2023-12-04T05:39:21.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.648+0000 7f72b5ee9640 1 -- 172.21.15.111:0/2670128861 learned_addr learned my addr 172.21.15.111:0/2670128861 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8151bce640 1 -- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f814c0d09c0 msgr2=0x7f814c149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8151bce640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f814c0d09c0 0x7f814c149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8151bce640 1 -- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f814c0d05e0 msgr2=0x7f814c1ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8151bce640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f814c0d05e0 0x7f814c1ed860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8151bce640 1 -- 172.21.15.111:0/2170322232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009a70 con 0x7f814c1eca50 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f8150bcc640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f814c0d09c0 0x7f814c149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c0b34b0 0x7f9a4c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c0cd440 0x7f9a4c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4b59e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1f7900 con 0x7f9a4c071940 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c14eb50 con 0x7f9a4c0cd440 2023-12-04T05:39:21.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1f7d90 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7fc35659c640 1 -- 172.21.15.111:0/3689049521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3500380e0 msgr2=0x7fc3501809d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f990c8b3640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f990806aed0 0x7f9908150050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f990c8b3640 1 -- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 msgr2=0x7f99081519f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:39:21.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f990c8b3640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99080b4f50 0x7f99081519f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f990c8b3640 1 -- 172.21.15.111:0/508173389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc009400 con 0x7f9908161680 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f72b4ee7640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b0203980 0x7f72b015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f72b5ee9640 1 -- 172.21.15.111:0/2670128861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b0203980 msgr2=0x7f72b015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72b0203980 0x7f72b015c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f72b5ee9640 1 -- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c005280 msgr2=0x7f72b016d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c005280 0x7f72b016d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f72b5ee9640 1 -- 172.21.15.111:0/2670128861 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b015d3e0 con 0x7f72b015a860 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f451ffff640 1 -- 172.21.15.111:0/1064648371 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518031830 con 0x7f452806bf20 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f451ffff640 1 -- 172.21.15.111:0/1064648371 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518031990 con 0x7f452806bf20 2023-12-04T05:39:21.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f451ffff640 1 -- 172.21.15.111:0/1064648371 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518031d20 con 0x7f452806bf20 2023-12-04T05:39:21.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66c0d79640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc1feec0 0x7f66bc070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f84e72a8640 1 -- 172.21.15.111:0/2770905739 shutdown_connections 2023-12-04T05:39:21.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f66bbfff640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc0cc800 0x7f66bc157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f84e72a8640 1 -- 172.21.15.111:0/2770905739 wait complete. 2023-12-04T05:39:21.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7fc35659c640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3500380e0 0x7fc3501809d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7fc35659c640 1 -- 172.21.15.111:0/3689049521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3500eacd0 msgr2=0x7fc350049700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7fc35659c640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3500eacd0 0x7fc350049700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7fc35659c640 1 -- 172.21.15.111:0/3689049521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0094d0 con 0x7fc35004ffa0 2023-12-04T05:39:21.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7fc35759e640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3500eacd0 0x7fc350049700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:21.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f72b5ee9640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72b015a860 0x7f72b015fda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f72a8001680 tx=0x7f72a80014a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a49d9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c0b34b0 0x7f9a4c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a49d9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c0b34b0 0x7f9a4c0cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40070/0 (socket says 172.21.15.111:40070) 2023-12-04T05:39:21.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1724072168 learned_addr learned my addr 172.21.15.111:0/1724072168 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.649+0000 7f9a4ad9d640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c0cd440 0x7f9a4c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66bb7fe640 1 -- 172.21.15.111:0/1105124951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc0cc800 msgr2=0x7f66bc157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66bb7fe640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc0cc800 0x7f66bc157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66bb7fe640 1 -- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc1feec0 msgr2=0x7f66bc070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66bb7fe640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc1feec0 0x7f66bc070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66bb7fe640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009830 con 0x7f66bc153660 2023-12-04T05:39:21.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f990c8b3640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9908161680 0x7f99080cec90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f990000cb10 tx=0x7f990000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7fc35659c640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc35004ffa0 0x7fc3500dce80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc35802cf20 tx=0x7fc358067f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8039070 con 0x7f72b015a860 2023-12-04T05:39:21.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a4a59c640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a4c071940 0x7f9a4c14b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66bb7fe640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc153660 0x7f66bc155ed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02d8d0 tx=0x7f66ac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0353f0 con 0x7f66bc153660 2023-12-04T05:39:21.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a803a040 con 0x7f72b015a860 2023-12-04T05:39:21.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000ac70 con 0x7f9908161680 2023-12-04T05:39:21.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990806e6f0 con 0x7f9908161680 2023-12-04T05:39:21.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c0cd440 msgr2=0x7f9a4c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a49d9b640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c0cd440 0x7f9a4c0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a4c071940 msgr2=0x7f9a4c14b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a49d9b640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a4c071940 0x7f9a4c14b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a49d9b640 1 -- 172.21.15.111:0/1724072168 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44009b30 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9a4a59c640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a4c071940 0x7f9a4c14b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:21.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806bf20 msgr2=0x7f452806de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f452f3ce640 1 --2- 172.21.15.111:0/1064648371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806bf20 0x7f452806de20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45180097a0 tx=0x7f451802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35806bc80 con 0x7fc35004ffa0 2023-12-04T05:39:21.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac03e8b0 con 0x7f66bc153660 2023-12-04T05:39:21.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac03ec60 con 0x7f66bc153660 2023-12-04T05:39:21.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990000add0 con 0x7f9908161680 2023-12-04T05:39:21.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900012620 con 0x7f9908161680 2023-12-04T05:39:21.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f8151bce640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c1eca50 0x7f814c15ae00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f814400efc0 tx=0x7f814400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99080ce770 con 0x7f9908161680 2023-12-04T05:39:21.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc350181140 con 0x7fc35004ffa0 2023-12-04T05:39:21.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc350181700 con 0x7fc35004ffa0 2023-12-04T05:39:21.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358071050 con 0x7fc35004ffa0 2023-12-04T05:39:21.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 shutdown_connections 2023-12-04T05:39:21.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f452f3ce640 1 --2- 172.21.15.111:0/1064648371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806bf20 0x7f452806de20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 >> 172.21.15.111:0/1064648371 conn(0x7f45280b1c40 msgr2=0x7f45280b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 shutdown_connections 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f452f3ce640 1 -- 172.21.15.111:0/1064648371 wait complete. 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc156cc0 con 0x7f66bc153660 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.650+0000 7f72b794c640 1 -- 172.21.15.111:0/2670128861 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b016c650 con 0x7f72b015a860 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f72b794c640 1 -- 172.21.15.111:0/2670128861 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b016cc10 con 0x7f72b015a860 2023-12-04T05:39:21.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80079d0 con 0x7f72b015a860 2023-12-04T05:39:21.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f990806e570 con 0x7f9908161680 2023-12-04T05:39:21.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f84e72a8640 1 Processor -- start 2023-12-04T05:39:21.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f84e72a8640 1 -- start start 2023-12-04T05:39:21.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3580936b0 con 0x7fc35004ffa0 2023-12-04T05:39:21.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f84e72a8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004f90 0x7f84d804e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f84e72a8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84d80c39f0 0x7f84d80c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f84e72a8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84d80c6e70 0x7f84d80c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f84e72a8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d804dc80 con 0x7f84d80c6e70 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f84e72a8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80c43b0 con 0x7f84d80c39f0 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f84e72a8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80c7830 con 0x7f84cc004f90 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc156410 con 0x7f66bc153660 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.651+0000 7f9a49d9b640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c0b34b0 0x7f9a4c0cbd90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009fa0 tx=0x7f9a44004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4403e660 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4403e7e0 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4403eb90 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66bc0c3ec0 con 0x7f66bc153660 2023-12-04T05:39:21.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc338000fc0 con 0x7fc35004ffa0 2023-12-04T05:39:21.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9900007070 con 0x7f9908161680 2023-12-04T05:39:21.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f84e62a6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004f90 0x7f84d804e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144009cd0 con 0x7f814c1eca50 2023-12-04T05:39:21.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.652+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814400f040 con 0x7f814c1eca50 2023-12-04T05:39:21.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144003990 con 0x7f814c1eca50 2023-12-04T05:39:21.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f452f3ce640 1 Processor -- start 2023-12-04T05:39:21.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f8153631640 1 -- 172.21.15.111:0/2170322232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c009400 con 0x7f814c1eca50 2023-12-04T05:39:21.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f72a8032070 con 0x7f72b015a860 2023-12-04T05:39:21.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f84e62a6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004f90 0x7f84d804e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40080/0 (socket says 172.21.15.111:40080) 2023-12-04T05:39:21.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f84e62a6640 1 -- 172.21.15.111:0/2958165400 learned_addr learned my addr 172.21.15.111:0/2958165400 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84d80c6e70 0x7f84d80c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f452f3ce640 1 -- start start 2023-12-04T05:39:21.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f8153631640 1 -- 172.21.15.111:0/2170322232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c15de10 con 0x7f814c1eca50 2023-12-04T05:39:21.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f72b794c640 1 -- 172.21.15.111:0/2670128861 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72b00b3f50 con 0x7f72b015a860 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e5aa5640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84d80c39f0 0x7f84d80c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e62a6640 1 -- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84d80c39f0 msgr2=0x7f84d80c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e62a6640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84d80c39f0 0x7f84d80c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e62a6640 1 -- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84d80c6e70 msgr2=0x7f84d80c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e62a6640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84d80c6e70 0x7f84d80c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e62a6640 1 -- 172.21.15.111:0/2958165400 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80c7d10 con 0x7f84cc004f90 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f66ac049460 con 0x7f66bc153660 2023-12-04T05:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e5aa5640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84d80c39f0 0x7f84d80c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:21.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f72a5ffb640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f729406b370 0x7f729406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f9904ff9640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f98ec06b2a0 0x7f98ec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9900089b90 con 0x7f9908161680 2023-12-04T05:39:21.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f9904ff9640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] conn(0x7f98ec0734c0 0x7f98ec0758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 --> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98ec075f90 con 0x7f98ec0734c0 2023-12-04T05:39:21.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.653+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f990008e2a0 con 0x7f9908161680 2023-12-04T05:39:21.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c1f6fd0 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84e62a6640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004f90 0x7f84d804e0a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c5c0 tx=0x7f84dc00ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c1f7590 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc011800 con 0x7f84cc004f90 2023-12-04T05:39:21.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc011960 con 0x7f84cc004f90 2023-12-04T05:39:21.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc011800 con 0x7f84cc004f90 2023-12-04T05:39:21.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f72b56e8640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f729406b370 0x7f729406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72a80ab150 con 0x7f72b015a860 2023-12-04T05:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f72b56e8640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f729406b370 0x7f729406d830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72b016b660 tx=0x7f72ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f814c1624b0 con 0x7f814c1eca50 2023-12-04T05:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f72a5ffb640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] conn(0x7f7294073420 0x7f7294075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f66b8ff9640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f669c06b370 0x7f669c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66ac0acc20 con 0x7f66bc153660 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f66b8ff9640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] conn(0x7f669c073510 0x7f669c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.654+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f669c075fe0 con 0x7f669c073510 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ac077c40 con 0x7f66bc153660 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 --> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7294075ef0 con 0x7f7294073420 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72a80ad6b0 con 0x7f72b015a860 2023-12-04T05:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f66bbfff640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f669c06b370 0x7f669c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d80d2a90 con 0x7f84cc004f90 2023-12-04T05:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d804d8a0 con 0x7f84cc004f90 2023-12-04T05:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f72b4ee7640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] conn(0x7f7294073420 0x7f7294075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f9907fff640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f98ec06b2a0 0x7f98ec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f66c0d79640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] conn(0x7f669c073510 0x7f669c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f99077fe640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] conn(0x7f98ec0734c0 0x7f98ec0758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.655+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a30000fc0 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f66bbfff640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f669c06b370 0x7f669c06d830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66b4006fd0 tx=0x7f66b4009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f9907fff640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f98ec06b2a0 0x7f98ec06d760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009810 tx=0x7f98f4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f99077fe640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] conn(0x7f98ec0734c0 0x7f98ec0758e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452f3ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452806bf20 0x7f452815e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f72b4ee7640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] conn(0x7f7294073420 0x7f7294075840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452f3ce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806ee20 0x7f45280cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452f3ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45280cdcd0 0x7f45280d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9a4403ecf0 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452f3ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c6950 con 0x7f45280cdcd0 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452f3ce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281616b0 con 0x7f452806ee20 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452f3ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c58b0 con 0x7f452806bf20 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452d96b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45280cdcd0 0x7f45280d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452d96b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45280cdcd0 0x7f45280d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58768/0 (socket says 172.21.15.111:58768) 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452d16a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452806bf20 0x7f452815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7f452d96b640 1 -- 172.21.15.111:0/4129575859 learned_addr learned my addr 172.21.15.111:0/4129575859 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f452c969640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806ee20 0x7f45280cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a3affd640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9a2c06b370 0x7f9a2c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.656+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc358090070 con 0x7fc35004ffa0 2023-12-04T05:39:21.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f66c0d79640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] conn(0x7f669c073510 0x7f669c075930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== osd.4 v2:172.21.15.192:6804/465617341 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f98ec075f90 con 0x7f98ec0734c0 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a440ac660 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a4a59c640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9a2c06b370 0x7f9a2c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a3affd640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] conn(0x7f9a2c073590 0x7f9a2c0759b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 --> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a2c076060 con 0x7f9a2c073590 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a440aec80 con 0x7f9a4c0b34b0 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f84dc011ad0 con 0x7f84cc004f90 2023-12-04T05:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7fc33f7fe640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34406b370 0x7fc34406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f452d96b640 1 -- 172.21.15.111:0/4129575859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452806bf20 msgr2=0x7f452815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f452d96b640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452806bf20 0x7f452815e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f452d96b640 1 -- 172.21.15.111:0/4129575859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806ee20 msgr2=0x7f45280cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f452d96b640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452806ee20 0x7f45280cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f452d96b640 1 -- 172.21.15.111:0/4129575859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4518009400 con 0x7f45280cdcd0 2023-12-04T05:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc358103bd0 con 0x7fc35004ffa0 2023-12-04T05:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f84d6ffd640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84c406b390 0x7f84c406d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f84d80df6a0 con 0x7f84cc004f90 2023-12-04T05:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f9a4a59c640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9a2c06b370 0x7f9a2c06d830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009920 tx=0x7f9a40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8144014480 con 0x7f814c1eca50 2023-12-04T05:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7fc33f7fe640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] conn(0x7fc344073420 0x7fc344075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== osd.3 v2:172.21.15.111:6816/413639041 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7294075ef0 con 0x7f7294073420 2023-12-04T05:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 --> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc344075ef0 con 0x7fc344073420 2023-12-04T05:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84dc0896c0 con 0x7f84cc004f90 2023-12-04T05:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.657+0000 7f84e5aa5640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84c406b390 0x7f84c406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc358103f80 con 0x7fc35004ffa0 2023-12-04T05:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f84d6ffd640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] conn(0x7f84c4073530 0x7f84c4075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 --> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84c4076000 con 0x7f84c4073530 2023-12-04T05:39:21.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f452d96b640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45280cdcd0 0x7f45280d02a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f452000ca20 tx=0x7f452000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84dc011800 con 0x7f84cc004f90 2023-12-04T05:39:21.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f8141ffb640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f813006b3a0 0x7f813006d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f9a4ad9d640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] conn(0x7f9a2c073590 0x7f9a2c0759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000abd0 con 0x7f45280cdcd0 2023-12-04T05:39:21.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== osd.2 v2:172.21.15.111:6817/306971668 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f669c075fe0 con 0x7f669c073510 2023-12-04T05:39:21.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] conn(0x7f84c4073530 0x7f84c4075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f81513cd640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f813006b3a0 0x7f813006d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f814408a6f0 con 0x7f814c1eca50 2023-12-04T05:39:21.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f8141ffb640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] conn(0x7f81300735c0 0x7f81300759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 --> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8130076090 con 0x7f81300735c0 2023-12-04T05:39:21.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f814408cd10 con 0x7f814c1eca50 2023-12-04T05:39:21.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f9a4ad9d640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] conn(0x7f9a2c073590 0x7f9a2c0759b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f84e6aa7640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] conn(0x7f84c4073530 0x7f84c4075950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f84e5aa5640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84c406b390 0x7f84c406d850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d80d2630 tx=0x7f84d0011040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f8150bcc640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] conn(0x7f81300735c0 0x7f81300759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452000ad30 con 0x7f45280cdcd0 2023-12-04T05:39:21.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f81513cd640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f813006b3a0 0x7f813006d860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8148001eb0 tx=0x7f8148001a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.658+0000 7fc356d9d640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34406b370 0x7fc34406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7fc356d9d640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34406b370 0x7fc34406d830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc34c02b9c0 tx=0x7fc34c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7fc35759e640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] conn(0x7fc344073420 0x7fc344075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520012620 con 0x7f45280cdcd0 2023-12-04T05:39:21.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f452f3ce640 1 -- 172.21.15.111:0/4129575859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45280c5b90 con 0x7f45280cdcd0 2023-12-04T05:39:21.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f452f3ce640 1 -- 172.21.15.111:0/4129575859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45280c6150 con 0x7f45280cdcd0 2023-12-04T05:39:21.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== osd.5 v2:172.21.15.192:6800/659116358 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9a2c076060 con 0x7f9a2c073590 2023-12-04T05:39:21.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.660+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== osd.6 v2:172.21.15.192:6816/363847578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f84c4076000 con 0x7f84c4073530 2023-12-04T05:39:21.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.660+0000 7fc35759e640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] conn(0x7fc344073420 0x7fc344075840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.659+0000 7f8150bcc640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] conn(0x7f81300735c0 0x7f81300759e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.660+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== osd.0 v2:172.21.15.111:6800/378032548 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc344075ef0 con 0x7fc344073420 2023-12-04T05:39:21.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.661+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== osd.7 v2:172.21.15.192:6818/1243890085 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8130076090 con 0x7f81300735c0 2023-12-04T05:39:21.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.661+0000 7f452f3ce640 1 -- 172.21.15.111:0/4129575859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4528071ed0 con 0x7f45280cdcd0 2023-12-04T05:39:21.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.662+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4520007070 con 0x7f45280cdcd0 2023-12-04T05:39:21.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.662+0000 7f451dffb640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f450c06b2a0 0x7f450c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.663+0000 7f452d16a640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f450c06b2a0 0x7f450c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.663+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4520089b60 con 0x7f45280cdcd0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.663+0000 7f451dffb640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] conn(0x7f450c0734c0 0x7f450c0758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.663+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 --> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f450c075f90 con 0x7f450c0734c0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.663+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f452008e2a0 con 0x7f45280cdcd0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.663+0000 7f452c969640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] conn(0x7f450c0734c0 0x7f450c0758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.664+0000 7f452d16a640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f450c06b2a0 0x7f450c06d760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4524004ea0 tx=0x7f4524009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.664+0000 7f452c969640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] conn(0x7f450c0734c0 0x7f450c0758e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.665+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== osd.1 v2:172.21.15.111:6808/909770062 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f450c075f90 con 0x7f450c0734c0 2023-12-04T05:39:21.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.689+0000 7f66c157a640 1 -- 172.21.15.111:0/1105124951 --> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f66bc156e50 con 0x7f669c073510 2023-12-04T05:39:21.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.690+0000 7f72b794c640 1 -- 172.21.15.111:0/2670128861 --> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72b00cc830 con 0x7f7294073420 2023-12-04T05:39:21.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.690+0000 7f66b8ff9640 1 -- 172.21.15.111:0/1105124951 <== osd.2 v2:172.21.15.111:6817/306971668 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f66bc156e50 con 0x7f669c073510 2023-12-04T05:39:21.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.690+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 --> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9908161a60 con 0x7f98ec0734c0 2023-12-04T05:39:21.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.690+0000 7f72a5ffb640 1 -- 172.21.15.111:0/2670128861 <== osd.3 v2:172.21.15.111:6816/413639041 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f72a000b750 con 0x7f7294073420 2023-12-04T05:39:21.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] conn(0x7f669c073510 msgr2=0x7f669c075930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66b27fc640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6817/306971668,v1:172.21.15.111:6819/306971668] conn(0x7f669c073510 0x7f669c075930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] conn(0x7f7294073420 msgr2=0x7f7294075840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6816/413639041,v1:172.21.15.111:6818/413639041] conn(0x7f7294073420 0x7f7294075840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f729406b370 msgr2=0x7f729406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f729406b370 0x7f729406d830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72b016b660 tx=0x7f72ac009040 comp rx=0 tx=0).stop 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72b015a860 msgr2=0x7f72b015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 --2- 172.21.15.111:0/2670128861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72b015a860 0x7f72b015fda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f72a8001680 tx=0x7f72a80014a0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72b5ee9640 1 -- 172.21.15.111:0/2670128861 reap_dead start 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 shutdown_connections 2023-12-04T05:39:21.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 >> 172.21.15.111:0/2670128861 conn(0x7f72b006c250 msgr2=0x7f72b00affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f669c06b370 msgr2=0x7f669c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66b27fc640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f669c06b370 0x7f669c06d830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66b4006fd0 tx=0x7f66b4009060 comp rx=0 tx=0).stop 2023-12-04T05:39:21.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc153660 msgr2=0x7f66bc155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66b27fc640 1 --2- 172.21.15.111:0/1105124951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc153660 0x7f66bc155ed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02d8d0 tx=0x7f66ac004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:21.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f66c0d79640 1 -- 172.21.15.111:0/1105124951 reap_dead start 2023-12-04T05:39:21.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 --> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f84d80df6a0 con 0x7f84c4073530 2023-12-04T05:39:21.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.691+0000 7f9904ff9640 1 -- 172.21.15.111:0/508173389 <== osd.4 v2:172.21.15.192:6804/465617341 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98fc031b00 con 0x7f98ec0734c0 2023-12-04T05:39:21.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 shutdown_connections 2023-12-04T05:39:21.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f72937fe640 1 -- 172.21.15.111:0/2670128861 wait complete. 2023-12-04T05:39:21.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 shutdown_connections 2023-12-04T05:39:21.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 >> 172.21.15.111:0/1105124951 conn(0x7f66bc06a8b0 msgr2=0x7f66bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 --> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc338002cb0 con 0x7fc344073420 2023-12-04T05:39:21.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f84d6ffd640 1 -- 172.21.15.111:0/2958165400 <== osd.6 v2:172.21.15.192:6816/363847578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f84e004f770 con 0x7f84c4073530 2023-12-04T05:39:21.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 shutdown_connections 2023-12-04T05:39:21.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f66b27fc640 1 -- 172.21.15.111:0/1105124951 wait complete. 2023-12-04T05:39:21.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] conn(0x7f84c4073530 msgr2=0x7f84c4075950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f84e72a8640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:6816/363847578,v1:172.21.15.192:6817/363847578] conn(0x7f84c4073530 0x7f84c4075950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7fc33f7fe640 1 -- 172.21.15.111:0/3689049521 <== osd.0 v2:172.21.15.111:6800/378032548 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc338002cb0 con 0x7fc344073420 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] conn(0x7f98ec0734c0 msgr2=0x7f98ec0758e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990e316640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.192:6804/465617341,v1:172.21.15.192:6806/465617341] conn(0x7f98ec0734c0 0x7f98ec0758e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f98ec06b2a0 msgr2=0x7f98ec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990e316640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f98ec06b2a0 0x7f98ec06d760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009810 tx=0x7f98f4009190 comp rx=0 tx=0).stop 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9908161680 msgr2=0x7f99080cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990e316640 1 --2- 172.21.15.111:0/508173389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9908161680 0x7f99080cec90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f990000cb10 tx=0x7f990000a060 comp rx=0 tx=0).stop 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.692+0000 7f990c8b3640 1 -- 172.21.15.111:0/508173389 reap_dead start 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 shutdown_connections 2023-12-04T05:39:21.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 >> 172.21.15.111:0/508173389 conn(0x7f99080b1960 msgr2=0x7f99080aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 --> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f814c0693d0 con 0x7f81300735c0 2023-12-04T05:39:21.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84c406b390 msgr2=0x7f84c406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84c406b390 0x7f84c406d850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d80d2630 tx=0x7f84d0011040 comp rx=0 tx=0).stop 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004f90 msgr2=0x7f84d804e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 --2- 172.21.15.111:0/2958165400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84cc004f90 0x7f84d804e0a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c5c0 tx=0x7f84dc00ca80 comp rx=0 tx=0).stop 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e6aa7640 1 -- 172.21.15.111:0/2958165400 reap_dead start 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 shutdown_connections 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 >> 172.21.15.111:0/2958165400 conn(0x7f84d8009f80 msgr2=0x7f84d80dc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] conn(0x7fc344073420 msgr2=0x7fc344075840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7fc35c82c640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6800/378032548,v1:172.21.15.111:6801/378032548] conn(0x7fc344073420 0x7fc344075840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 shutdown_connections 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f990e316640 1 -- 172.21.15.111:0/508173389 wait complete. 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.693+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 shutdown_connections 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34406b370 msgr2=0x7fc34406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34406b370 0x7fc34406d830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc34c02b9c0 tx=0x7fc34c008030 comp rx=0 tx=0).stop 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc35004ffa0 msgr2=0x7fc3500dce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 --2- 172.21.15.111:0/3689049521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc35004ffa0 0x7fc3500dce80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc35802cf20 tx=0x7fc358067f40 comp rx=0 tx=0).stop 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35759e640 1 -- 172.21.15.111:0/3689049521 reap_dead start 2023-12-04T05:39:21.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 shutdown_connections 2023-12-04T05:39:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 >> 172.21.15.111:0/3689049521 conn(0x7fc350009f80 msgr2=0x7fc350036400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7f84e72a8640 1 -- 172.21.15.111:0/2958165400 wait complete. 2023-12-04T05:39:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 --> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a30002cb0 con 0x7f9a2c073590 2023-12-04T05:39:21.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 shutdown_connections 2023-12-04T05:39:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7f8141ffb640 1 -- 172.21.15.111:0/2170322232 <== osd.7 v2:172.21.15.192:6818/1243890085 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f814c0693d0 con 0x7f81300735c0 2023-12-04T05:39:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7fc35c82c640 1 -- 172.21.15.111:0/3689049521 wait complete. 2023-12-04T05:39:21.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.694+0000 7f9a3affd640 1 -- 172.21.15.111:0/1724072168 <== osd.5 v2:172.21.15.192:6800/659116358 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a30002cb0 con 0x7f9a2c073590 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] conn(0x7f81300735c0 msgr2=0x7f81300759e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f812f7fe640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.192:6818/1243890085,v1:172.21.15.192:6820/1243890085] conn(0x7f81300735c0 0x7f81300759e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] conn(0x7f9a2c073590 msgr2=0x7f9a2c0759b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:6800/659116358,v1:172.21.15.192:6801/659116358] conn(0x7f9a2c073590 0x7f9a2c0759b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9a2c06b370 msgr2=0x7f9a2c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9a2c06b370 0x7f9a2c06d830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009920 tx=0x7f9a40008040 comp rx=0 tx=0).stop 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c0b34b0 msgr2=0x7f9a4c0cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 --2- 172.21.15.111:0/1724072168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a4c0b34b0 0x7f9a4c0cbd90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009fa0 tx=0x7f9a44004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:21.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4ad9d640 1 -- 172.21.15.111:0/1724072168 reap_dead start 2023-12-04T05:39:21.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f813006b3a0 msgr2=0x7f813006d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f812f7fe640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f813006b3a0 0x7f813006d860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8148001eb0 tx=0x7f8148001a50 comp rx=0 tx=0).stop 2023-12-04T05:39:21.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c1eca50 msgr2=0x7f814c15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f812f7fe640 1 --2- 172.21.15.111:0/2170322232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f814c1eca50 0x7f814c15ae00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f814400efc0 tx=0x7f814400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:21.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 shutdown_connections 2023-12-04T05:39:21.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.695+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 >> 172.21.15.111:0/1724072168 conn(0x7f9a4c0b0950 msgr2=0x7f9a4c0cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.696+0000 7f8151bce640 1 -- 172.21.15.111:0/2170322232 reap_dead start 2023-12-04T05:39:21.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.696+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 shutdown_connections 2023-12-04T05:39:21.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.696+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 >> 172.21.15.111:0/2170322232 conn(0x7f814c06a8b0 msgr2=0x7f814c0b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.696+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 shutdown_connections 2023-12-04T05:39:21.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.696+0000 7f9a4b59e640 1 -- 172.21.15.111:0/1724072168 wait complete. 2023-12-04T05:39:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.697+0000 7f452f3ce640 1 -- 172.21.15.111:0/4129575859 --> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f452816c200 con 0x7f450c0734c0 2023-12-04T05:39:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.696+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 shutdown_connections 2023-12-04T05:39:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.697+0000 7f812f7fe640 1 -- 172.21.15.111:0/2170322232 wait complete. 2023-12-04T05:39:21.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.698+0000 7f451dffb640 1 -- 172.21.15.111:0/4129575859 <== osd.1 v2:172.21.15.111:6808/909770062 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4518031b00 con 0x7f450c0734c0 2023-12-04T05:39:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.699+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] conn(0x7f450c0734c0 msgr2=0x7f450c0758e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.699+0000 7f450b7fe640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6808/909770062,v1:172.21.15.111:6809/909770062] conn(0x7f450c0734c0 0x7f450c0758e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.699+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f450c06b2a0 msgr2=0x7f450c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.699+0000 7f450b7fe640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f450c06b2a0 0x7f450c06d760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4524004ea0 tx=0x7f4524009040 comp rx=0 tx=0).stop 2023-12-04T05:39:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45280cdcd0 msgr2=0x7f45280d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f450b7fe640 1 --2- 172.21.15.111:0/4129575859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45280cdcd0 0x7f45280d02a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f452000ca20 tx=0x7f452000cee0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f452d96b640 1 -- 172.21.15.111:0/4129575859 reap_dead start 2023-12-04T05:39:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 shutdown_connections 2023-12-04T05:39:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 >> 172.21.15.111:0/4129575859 conn(0x7f45280b1c40 msgr2=0x7f452806acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 shutdown_connections 2023-12-04T05:39:21.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.700+0000 7f450b7fe640 1 -- 172.21.15.111:0/4129575859 wait complete. 2023-12-04T05:39:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2023-12-04T05:39:21.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-04T05:39:21.758 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:21.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-04T05:39:21.759 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:21.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-04T05:39:21.760 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:21.760 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-04T05:39:21.760 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2023-12-04T05:39:21.761 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-04T05:39:21.761 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:21.762 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-04T05:39:21.762 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2023-12-04T05:39:21.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-04T05:39:21.764 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:21.764 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-04T05:39:21.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.963+0000 7f329810c640 1 Processor -- start 2023-12-04T05:39:21.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.963+0000 7f329810c640 1 -- start start 2023-12-04T05:39:21.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.964+0000 7f09a99db640 1 Processor -- start 2023-12-04T05:39:21.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.964+0000 7f09a99db640 1 -- start start 2023-12-04T05:39:21.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.964+0000 7f329810c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f32900aed80 0x7f329015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.964+0000 7f329810c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3290159ca0 con 0x7f32900662b0 2023-12-04T05:39:21.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.964+0000 7f329810c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32900b3160 con 0x7f329015aa60 2023-12-04T05:39:21.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.964+0000 7f329810c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32900b2f40 con 0x7f32900aed80 2023-12-04T05:39:21.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f916ff1d640 1 Processor -- start 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f32966a9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f32900662b0 0x7f3290070c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58678/0 (socket says 172.21.15.111:58678) 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f32966a9640 1 -- 172.21.15.111:0/2251194562 learned_addr learned my addr 172.21.15.111:0/2251194562 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f3295ea8640 1 --2- 172.21.15.111:0/2251194562 >> v2:172.21.15.192:3300/0 conn(0x7f32900aed80 0x7f329015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f3294ea6640 1 -- 172.21.15.111:0/2251194562 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224627046 0 0) 0x7f3290159ca0 con 0x7f32900662b0 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f3294ea6640 1 -- 172.21.15.111:0/2251194562 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f327c003520 con 0x7f32900662b0 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a99db640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c0e3080 0x7f099c057d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f3294ea6640 1 -- 172.21.15.111:0/2251194562 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 153350547 0 0) 0x7f327c003520 con 0x7f32900662b0 2023-12-04T05:39:21.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a99db640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f099c03e800 0x7f099c053bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f3295ea8640 1 -- 172.21.15.111:0/2251194562 >> v1:172.21.15.192:6789/0 conn(0x7f329015aa60 legacy=0x7f32900b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f3295ea8640 1 -- 172.21.15.111:0/2251194562 >> v1:172.21.15.111:6789/0 conn(0x7f32900662b0 legacy=0x7f3290070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a99db640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f099c0540f0 0x7f099c0e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f3295ea8640 1 -- 172.21.15.111:0/2251194562 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329015d4b0 con 0x7f32900aed80 2023-12-04T05:39:21.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a89d9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c0e3080 0x7f099c057d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a89d9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c0e3080 0x7f099c057d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40094/0 (socket says 172.21.15.111:40094) 2023-12-04T05:39:21.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a89d9640 1 -- 172.21.15.111:0/3690472505 learned_addr learned my addr 172.21.15.111:0/3690472505 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c054580 con 0x7f099c0540f0 2023-12-04T05:39:21.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c006160 con 0x7f099c03e800 2023-12-04T05:39:21.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c058290 con 0x7f099c0e3080 2023-12-04T05:39:21.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.965+0000 7f916ff1d640 1 -- start start 2023-12-04T05:39:21.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f916ff1d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f916806a4c0 0x7f916806a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f93e71fb640 1 Processor -- start 2023-12-04T05:39:21.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f916ff1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f916806f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f916ff1d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916806ae60 con 0x7f9168164b40 2023-12-04T05:39:21.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f916ff1d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91680b2f70 con 0x7f916806a4c0 2023-12-04T05:39:21.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f916ff1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91680b3210 con 0x7f916806eea0 2023-12-04T05:39:21.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f88ca3c5640 1 Processor -- start 2023-12-04T05:39:21.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f9f32c83640 1 Processor -- start 2023-12-04T05:39:21.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f88ca3c5640 1 -- start start 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f93e71fb640 1 -- start start 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a3fff640 1 --2- 172.21.15.111:0/3690472505 >> v2:172.21.15.192:3300/0 conn(0x7f099c03e800 0x7f099c053bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.966+0000 7f09a91da640 1 --2- 172.21.15.111:0/3690472505 >> v2:172.21.15.111:3300/0 conn(0x7f099c0540f0 0x7f099c0e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f09a91da640 1 -- 172.21.15.111:0/3690472505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c0e3080 msgr2=0x7f099c057d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f09a91da640 1 --2- 172.21.15.111:0/3690472505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c0e3080 0x7f099c057d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f09a91da640 1 -- 172.21.15.111:0/3690472505 >> v2:172.21.15.192:3300/0 conn(0x7f099c03e800 msgr2=0x7f099c053bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f09a91da640 1 --2- 172.21.15.111:0/3690472505 >> v2:172.21.15.192:3300/0 conn(0x7f099c03e800 0x7f099c053bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f09a91da640 1 -- 172.21.15.111:0/3690472505 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c058530 con 0x7f099c0540f0 2023-12-04T05:39:21.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f3295ea8640 1 --2- 172.21.15.111:0/2251194562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900aed80 0x7f329015df90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32800097a0 tx=0x7f328002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdc01a3226f81fe server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f3294ea6640 1 -- 172.21.15.111:0/2251194562 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880019f0 con 0x7f32900aed80 2023-12-04T05:39:21.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f3294ea6640 1 -- 172.21.15.111:0/2251194562 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328002e050 con 0x7f32900aed80 2023-12-04T05:39:21.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f3294ea6640 1 -- 172.21.15.111:0/2251194562 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280030730 con 0x7f32900aed80 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f916e4ba640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f916806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f916e4ba640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f916806f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40104/0 (socket says 172.21.15.111:40104) 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88ca3c5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f88bc0e77e0 0x7f88bc038240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88ca3c5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88bc006160 0x7f88bc0065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88ca3c5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc0e7bc0 con 0x7f88bc0eb2c0 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88ca3c5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc057ee0 con 0x7f88bc0e77e0 2023-12-04T05:39:21.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88ca3c5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0581c0 con 0x7f88bc006160 2023-12-04T05:39:21.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f9f32c83640 1 -- start start 2023-12-04T05:39:21.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e71fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0161a10 0x7f93e0165a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7fc465edf640 1 Processor -- start 2023-12-04T05:39:21.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f916e4ba640 1 -- 172.21.15.111:0/2303731816 learned_addr learned my addr 172.21.15.111:0/2303731816 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f916e4ba640 1 -- 172.21.15.111:0/2303731816 >> v2:172.21.15.192:3300/0 conn(0x7f916806a4c0 msgr2=0x7f916806a920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-04T05:39:21.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f916e4ba640 1 --2- 172.21.15.111:0/2303731816 >> v2:172.21.15.192:3300/0 conn(0x7f916806a4c0 0x7f916806a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.967+0000 7f916e4ba640 1 -- 172.21.15.111:0/2303731816 >> v1:172.21.15.111:6789/0 conn(0x7f9168164b40 legacy=0x7f9168165c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:39:21.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f916e4ba640 1 -- 172.21.15.111:0/2303731816 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680b34f0 con 0x7f916806eea0 2023-12-04T05:39:21.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e71fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f93e0144b80 0x7f93e01471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f32c83640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f2c0afea0 0x7f9f2c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f32c83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9f2c0b38d0 0x7f9f2c15d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f32c83640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c0b2130 con 0x7f9f2c150280 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f32c83640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0b3da0 con 0x7f9f2c0b38d0 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f32c83640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0b4080 con 0x7f9f2c0afea0 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88c8962640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88bc006160 0x7f88bc0065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88c8962640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88bc006160 0x7f88bc0065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57120/0 (socket says 172.21.15.111:57120) 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f88c8962640 1 -- 172.21.15.111:0/3520921182 learned_addr learned my addr 172.21.15.111:0/3520921182 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f329810c640 1 -- 172.21.15.111:0/2251194562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900aed80 msgr2=0x7f329015df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f329810c640 1 --2- 172.21.15.111:0/2251194562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900aed80 0x7f329015df90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32800097a0 tx=0x7f328002b790 comp rx=0 tx=0).stop 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e71fb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e0162060 con 0x7f93e015b510 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e71fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e0165f50 con 0x7f93e0144b80 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e71fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e0071710 con 0x7f93e0161a10 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e61f9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f93e015b510 0x7f93e015b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58688/0 (socket says 172.21.15.111:58688) 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e61f9640 1 -- 172.21.15.111:0/4249966712 learned_addr learned my addr 172.21.15.111:0/4249966712 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e59f8640 1 --2- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0161a10 0x7f93e0165a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f93e69fa640 1 --2- 172.21.15.111:0/4249966712 >> v2:172.21.15.192:3300/0 conn(0x7f93e0144b80 0x7f93e01471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f2bfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f9f2c150280 0x7f9f2c0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43094/0 (socket says 172.21.15.111:43094) 2023-12-04T05:39:21.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7f9f2bfff640 1 -- 172.21.15.111:0/3916662962 learned_addr learned my addr 172.21.15.111:0/3916662962 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:21.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f88c3fff640 1 --2- 172.21.15.111:0/3520921182 >> v2:172.21.15.111:3300/0 conn(0x7f88bc0e77e0 0x7f88bc038240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f9f31220640 1 --2- 172.21.15.111:0/3916662962 >> v2:172.21.15.111:3300/0 conn(0x7f9f2c0b38d0 0x7f9f2c15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f9f30a1f640 1 --2- 172.21.15.111:0/3916662962 >> v2:172.21.15.192:3300/0 conn(0x7f9f2c0afea0 0x7f9f2c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.968+0000 7fc465edf640 1 -- start start 2023-12-04T05:39:21.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f88c8962640 1 -- 172.21.15.111:0/3520921182 >> v2:172.21.15.111:3300/0 conn(0x7f88bc0e77e0 msgr2=0x7f88bc038240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f88c8962640 1 --2- 172.21.15.111:0/3520921182 >> v2:172.21.15.111:3300/0 conn(0x7f88bc0e77e0 0x7f88bc038240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f88c8962640 1 -- 172.21.15.111:0/3520921182 >> v1:172.21.15.111:6789/0 conn(0x7f88bc0eb2c0 legacy=0x7f88bc0eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f88c8962640 1 -- 172.21.15.111:0/3520921182 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0584a0 con 0x7f88bc006160 2023-12-04T05:39:21.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f916e4ba640 1 --2- 172.21.15.111:0/2303731816 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f916806f330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e50 tx=0x7f915802bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0ae7f54ddd8536 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f93e69fa640 1 -- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0161a10 msgr2=0x7f93e0165a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f329810c640 1 -- 172.21.15.111:0/2251194562 shutdown_connections 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f329810c640 1 --2- 172.21.15.111:0/2251194562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900aed80 0x7f329015df90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f329810c640 1 -- 172.21.15.111:0/2251194562 >> 172.21.15.111:0/2251194562 conn(0x7f32900cf600 msgr2=0x7f32900cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f329810c640 1 -- 172.21.15.111:0/2251194562 shutdown_connections 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f329810c640 1 -- 172.21.15.111:0/2251194562 wait complete. 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7fc465edf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc4600c4480 0x7fc4600cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7fc465edf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4600ce790 con 0x7fc460161800 2023-12-04T05:39:21.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc465edf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4600ce4f0 con 0x7fc4600cdd20 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc465edf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc460162bf0 con 0x7fc4600c4480 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f9f31220640 1 -- 172.21.15.111:0/3916662962 >> v2:172.21.15.192:3300/0 conn(0x7f9f2c0afea0 msgr2=0x7f9f2c15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f9f31220640 1 --2- 172.21.15.111:0/3916662962 >> v2:172.21.15.192:3300/0 conn(0x7f9f2c0afea0 0x7f9f2c15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f9f31220640 1 -- 172.21.15.111:0/3916662962 >> v1:172.21.15.192:6789/0 conn(0x7f9f2c150280 legacy=0x7f9f2c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f9f31220640 1 -- 172.21.15.111:0/3916662962 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0b4360 con 0x7f9f2c0b38d0 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f93e69fa640 1 --2- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0161a10 0x7f93e0165a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f93e69fa640 1 -- 172.21.15.111:0/4249966712 >> v1:172.21.15.111:6789/0 conn(0x7f93e015b510 legacy=0x7f93e015b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.969+0000 7f93e69fa640 1 -- 172.21.15.111:0/4249966712 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0162340 con 0x7f93e0144b80 2023-12-04T05:39:21.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f9573fff640 1 Processor -- start 2023-12-04T05:39:21.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f09a91da640 1 --2- 172.21.15.111:0/3690472505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 0x7f099c0e6bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09a40692c0 tx=0x7f09a40a5dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517153530866e921 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc45ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc4600c4480 0x7fc4600cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc45ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc4600c4480 0x7fc4600cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58804/0 (socket says 172.21.15.111:58804) 2023-12-04T05:39:21.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc45ffff640 1 -- 172.21.15.111:0/2260675492 learned_addr learned my addr 172.21.15.111:0/2260675492 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f916ccb7640 1 -- 172.21.15.111:0/2303731816 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915802caa0 con 0x7f916806eea0 2023-12-04T05:39:21.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f916ccb7640 1 -- 172.21.15.111:0/2303731816 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915802cc00 con 0x7f916806eea0 2023-12-04T05:39:21.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f916ccb7640 1 -- 172.21.15.111:0/2303731816 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580343f0 con 0x7f916806eea0 2023-12-04T05:39:21.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f88c8962640 1 --2- 172.21.15.111:0/3520921182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc006160 0x7f88bc0065f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88c4068b70 tx=0x7f88c40a5e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbba6825a7305abd server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f93e69fa640 1 --2- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0144b80 0x7f93e01471d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009d90 tx=0x7f93d802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef39d497be84c1d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f9573fff640 1 -- start start 2023-12-04T05:39:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f88c2ffd640 1 -- 172.21.15.111:0/3520921182 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40a6cc0 con 0x7f88bc006160 2023-12-04T05:39:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f9f31220640 1 --2- 172.21.15.111:0/3916662962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0b38d0 0x7f9f2c15d140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c06e3e0 tx=0x7f9f2002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed5372d44e005ff server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f88c2ffd640 1 -- 172.21.15.111:0/3520921182 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c40a6e60 con 0x7f88bc006160 2023-12-04T05:39:21.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f88c2ffd640 1 -- 172.21.15.111:0/3520921182 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40ae640 con 0x7f88bc006160 2023-12-04T05:39:21.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7f9f2b7fe640 1 -- 172.21.15.111:0/3916662962 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20030610 con 0x7f9f2c0b38d0 2023-12-04T05:39:21.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f9f2b7fe640 1 -- 172.21.15.111:0/3916662962 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20030770 con 0x7f9f2c0b38d0 2023-12-04T05:39:21.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f93e51f7640 1 -- 172.21.15.111:0/4249966712 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d802cc50 con 0x7f93e0144b80 2023-12-04T05:39:21.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f9f2b7fe640 1 -- 172.21.15.111:0/3916662962 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20030b00 con 0x7f9f2c0b38d0 2023-12-04T05:39:21.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f93e51f7640 1 -- 172.21.15.111:0/4249966712 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d802cdb0 con 0x7f93e0144b80 2023-12-04T05:39:21.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f09a37fe640 1 -- 172.21.15.111:0/3690472505 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a40a6c40 con 0x7f099c0540f0 2023-12-04T05:39:21.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f09a37fe640 1 -- 172.21.15.111:0/3690472505 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a40a6da0 con 0x7f099c0540f0 2023-12-04T05:39:21.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f09a37fe640 1 -- 172.21.15.111:0/3690472505 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a40ae640 con 0x7f099c0540f0 2023-12-04T05:39:21.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f916ff1d640 1 -- 172.21.15.111:0/2303731816 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 msgr2=0x7f916806f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f916ff1d640 1 --2- 172.21.15.111:0/2303731816 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f916806f330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e50 tx=0x7f915802bf00 comp rx=0 tx=0).stop 2023-12-04T05:39:21.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f9573fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9574165a10 0x7f95741471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:21.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f93e51f7640 1 -- 172.21.15.111:0/4249966712 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8034660 con 0x7f93e0144b80 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f9573fff640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9574165e80 con 0x7f9574161a10 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f9573fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9574162060 con 0x7f957415b510 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f9573fff640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9574071710 con 0x7f9574165a10 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f95737fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9574165a10 0x7f95741471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f95737fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9574165a10 0x7f95741471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58820/0 (socket says 172.21.15.111:58820) 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.971+0000 7f95737fe640 1 -- 172.21.15.111:0/1994314558 learned_addr learned my addr 172.21.15.111:0/1994314558 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:21.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 -- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0144b80 msgr2=0x7f93e01471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 --2- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0144b80 0x7f93e01471d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009d90 tx=0x7f93d802bea0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f329810c640 1 Processor -- start 2023-12-04T05:39:21.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 msgr2=0x7f099c0e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 --2- 172.21.15.111:0/3690472505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 0x7f099c0e6bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09a40692c0 tx=0x7f09a40a5dd0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc45ffff640 1 -- 172.21.15.111:0/2260675492 >> v1:172.21.15.192:6789/0 conn(0x7fc4600cdd20 legacy=0x7fc4600cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc45ffff640 1 -- 172.21.15.111:0/2260675492 >> v1:172.21.15.111:6789/0 conn(0x7fc460161800 legacy=0x7fc4600c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.970+0000 7fc45ffff640 1 -- 172.21.15.111:0/2260675492 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4600cb590 con 0x7fc4600c4480 2023-12-04T05:39:21.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 -- 172.21.15.111:0/4249966712 shutdown_connections 2023-12-04T05:39:21.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3520921182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc006160 msgr2=0x7f88bc0065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 --2- 172.21.15.111:0/3520921182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc006160 0x7f88bc0065f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88c4068b70 tx=0x7f88c40a5e50 comp rx=0 tx=0).stop 2023-12-04T05:39:21.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f329810c640 1 -- start start 2023-12-04T05:39:21.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f95737fe640 1 -- 172.21.15.111:0/1994314558 >> v1:172.21.15.192:6789/0 conn(0x7f957415b510 legacy=0x7f957415b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-04T05:39:21.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f95737fe640 1 -- 172.21.15.111:0/1994314558 >> v1:172.21.15.111:6789/0 conn(0x7f9574161a10 legacy=0x7f9574144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:21.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f95737fe640 1 -- 172.21.15.111:0/1994314558 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574162340 con 0x7f9574165a10 2023-12-04T05:39:21.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3520921182 shutdown_connections 2023-12-04T05:39:21.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f9f32c83640 1 -- 172.21.15.111:0/3916662962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0b38d0 msgr2=0x7f9f2c15d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:21.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f9f32c83640 1 --2- 172.21.15.111:0/3916662962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0b38d0 0x7f9f2c15d140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c06e3e0 tx=0x7f9f2002b960 comp rx=0 tx=0).stop 2023-12-04T05:39:21.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 --2- 172.21.15.111:0/3520921182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc006160 0x7f88bc0065f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 --2- 172.21.15.111:0/3520921182 >> v2:172.21.15.111:3300/0 conn(0x7f88bc0e77e0 0x7f88bc038240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3520921182 >> 172.21.15.111:0/3520921182 conn(0x7f88bc0559a0 msgr2=0x7f88bc055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f916ff1d640 1 -- 172.21.15.111:0/2303731816 shutdown_connections 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f916ff1d640 1 --2- 172.21.15.111:0/2303731816 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f916806f330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f916ff1d640 1 --2- 172.21.15.111:0/2303731816 >> v2:172.21.15.192:3300/0 conn(0x7f916806a4c0 0x7f916806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f916ff1d640 1 -- 172.21.15.111:0/2303731816 >> 172.21.15.111:0/2303731816 conn(0x7f91680cfec0 msgr2=0x7f91680d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 shutdown_connections 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 --2- 172.21.15.111:0/3690472505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 0x7f099c0e6bf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 --2- 172.21.15.111:0/3690472505 >> v2:172.21.15.192:3300/0 conn(0x7f099c03e800 0x7f099c053bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 --2- 172.21.15.111:0/3690472505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c0e3080 0x7f099c057d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 >> 172.21.15.111:0/3690472505 conn(0x7f099c0559a0 msgr2=0x7f099c055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7fc45ffff640 1 --2- 172.21.15.111:0/2260675492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600c4480 0x7fc4600cd7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e50 tx=0x7fc45802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bead7091e0118b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:21.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 --2- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0144b80 0x7f93e01471d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 shutdown_connections 2023-12-04T05:39:21.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f09a99db640 1 -- 172.21.15.111:0/3690472505 wait complete. 2023-12-04T05:39:21.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f916ff1d640 1 -- 172.21.15.111:0/2303731816 shutdown_connections 2023-12-04T05:39:21.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3520921182 shutdown_connections 2023-12-04T05:39:21.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f916ff1d640 1 -- 172.21.15.111:0/2303731816 wait complete. 2023-12-04T05:39:21.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3520921182 wait complete. 2023-12-04T05:39:21.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 --2- 172.21.15.111:0/4249966712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0161a10 0x7f93e0165a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:21.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 -- 172.21.15.111:0/4249966712 >> 172.21.15.111:0/4249966712 conn(0x7f93e006a8b0 msgr2=0x7f93e006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:21.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 -- 172.21.15.111:0/4249966712 shutdown_connections 2023-12-04T05:39:21.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.972+0000 7f93e71fb640 1 -- 172.21.15.111:0/4249966712 wait complete. 2023-12-04T05:39:21.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7fc45f7fe640 1 -- 172.21.15.111:0/2260675492 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45802cb20 con 0x7fc4600c4480 2023-12-04T05:39:21.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7fc45f7fe640 1 -- 172.21.15.111:0/2260675492 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45802cc80 con 0x7fc4600c4480 2023-12-04T05:39:21.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7fc45f7fe640 1 -- 172.21.15.111:0/2260675492 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580343f0 con 0x7fc4600c4480 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f95737fe640 1 --2- 172.21.15.111:0/1994314558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f95741471d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f956c009e50 tx=0x7f956c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954c7151461964b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9571ffb640 1 -- 172.21.15.111:0/1994314558 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c02ca80 con 0x7f9574165a10 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9571ffb640 1 -- 172.21.15.111:0/1994314558 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c02cbe0 con 0x7f9574165a10 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9f32c83640 1 -- 172.21.15.111:0/3916662962 shutdown_connections 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9f32c83640 1 --2- 172.21.15.111:0/3916662962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0b38d0 0x7f9f2c15d140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9f32c83640 1 --2- 172.21.15.111:0/3916662962 >> v2:172.21.15.192:3300/0 conn(0x7f9f2c0afea0 0x7f9f2c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9f32c83640 1 -- 172.21.15.111:0/3916662962 >> 172.21.15.111:0/3916662962 conn(0x7f9f2c0aea80 msgr2=0x7f9f2c072250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9571ffb640 1 -- 172.21.15.111:0/1994314558 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0343f0 con 0x7f9574165a10 2023-12-04T05:39:22.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f329810c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900662b0 0x7f329014bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f329810c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32900aed80 0x7f3290144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 -- 172.21.15.111:0/2260675492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600c4480 msgr2=0x7fc4600cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f329810c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f329015aa60 0x7f329014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 --2- 172.21.15.111:0/2260675492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600c4480 0x7fc4600cd7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e50 tx=0x7fc45802bf80 comp rx=0 tx=0).stop 2023-12-04T05:39:22.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f329810c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329014a5c0 con 0x7f32900aed80 2023-12-04T05:39:22.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 -- 172.21.15.111:0/2260675492 shutdown_connections 2023-12-04T05:39:22.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 --2- 172.21.15.111:0/2260675492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600c4480 0x7fc4600cd7e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 -- 172.21.15.111:0/2260675492 >> 172.21.15.111:0/2260675492 conn(0x7fc4600d0da0 msgr2=0x7fc4600d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f32956a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32900aed80 0x7f3290144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f9573fff640 1 -- 172.21.15.111:0/1994314558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 msgr2=0x7f95741471d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f9573fff640 1 --2- 172.21.15.111:0/1994314558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f95741471d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f956c009e50 tx=0x7f956c02bee0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f32956a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32900aed80 0x7f3290144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58828/0 (socket says 172.21.15.111:58828) 2023-12-04T05:39:22.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f09a99db640 1 Processor -- start 2023-12-04T05:39:22.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f09a99db640 1 -- start start 2023-12-04T05:39:22.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f32956a7640 1 -- 172.21.15.111:0/165143087 learned_addr learned my addr 172.21.15.111:0/165143087 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f88ca3c5640 1 Processor -- start 2023-12-04T05:39:22.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9f32c83640 1 -- 172.21.15.111:0/3916662962 shutdown_connections 2023-12-04T05:39:22.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.973+0000 7f9f32c83640 1 -- 172.21.15.111:0/3916662962 wait complete. 2023-12-04T05:39:22.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f916ff1d640 1 Processor -- start 2023-12-04T05:39:22.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f916ff1d640 1 -- start start 2023-12-04T05:39:22.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f3295ea8640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900662b0 0x7f329014bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7f32966a9640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f329015aa60 0x7f329014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32901490c0 con 0x7f32900662b0 2023-12-04T05:39:22.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32901449d0 con 0x7f329015aa60 2023-12-04T05:39:22.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 -- 172.21.15.111:0/2260675492 shutdown_connections 2023-12-04T05:39:22.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.974+0000 7fc465edf640 1 -- 172.21.15.111:0/2260675492 wait complete. 2023-12-04T05:39:22.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f93e71fb640 1 Processor -- start 2023-12-04T05:39:22.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f93e71fb640 1 -- start start 2023-12-04T05:39:22.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f32966a9640 1 -- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900662b0 msgr2=0x7f329014bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f32966a9640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900662b0 0x7f329014bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f32966a9640 1 -- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32900aed80 msgr2=0x7f3290144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f32966a9640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32900aed80 0x7f3290144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f32966a9640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3280009400 con 0x7f329015aa60 2023-12-04T05:39:22.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f916ff1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 0x7f916813a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f916ff1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f9168150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a99db640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c03e800 0x7f099c188bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a99db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 0x7f099c051a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f32966a9640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f329015aa60 0x7f329014e7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3288001700 tx=0x7f328800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a99db640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c0e3080 0x7f099c0e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a99db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c00af90 con 0x7f099c0540f0 2023-12-04T05:39:22.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e71fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0144b80 0x7f93e00b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a89d9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c03e800 0x7f099c188bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a89d9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c03e800 0x7f099c188bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40140/0 (socket says 172.21.15.111:40140) 2023-12-04T05:39:22.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a89d9640 1 -- 172.21.15.111:0/2368140209 learned_addr learned my addr 172.21.15.111:0/2368140209 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e71fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93e015b510 0x7f93e01ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f88ca3c5640 1 -- start start 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88ca3c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88bc006160 0x7f88bc04f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88ca3c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 0x7f88bc00c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88ca3c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 0x7f88bc00ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88ca3c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00ac30 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88ca3c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00af10 con 0x7f88bc0e77e0 2023-12-04T05:39:22.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88ca3c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00b1f0 con 0x7f88bc006160 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88c8962640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 0x7f88bc00ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 0x7f88bc00c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c8962640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 0x7f88bc00ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58842/0 (socket says 172.21.15.111:58842) 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 0x7f88bc00c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57194/0 (socket says 172.21.15.111:57194) 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c37fe640 1 -- 172.21.15.111:0/3896281370 learned_addr learned my addr 172.21.15.111:0/3896281370 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c8962640 1 -- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88bc006160 msgr2=0x7f88bc04f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c8962640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88bc006160 0x7f88bc04f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a91da640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c0e3080 0x7f099c0e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c00b530 con 0x7f099c0e3080 2023-12-04T05:39:22.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c189130 con 0x7f099c03e800 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a91da640 1 -- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c03e800 msgr2=0x7f099c188bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a91da640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c03e800 0x7f099c188bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a91da640 1 -- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 msgr2=0x7f099c051a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a91da640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 0x7f099c051a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a91da640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a4068870 con 0x7f099c0e3080 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a91da640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c0e3080 0x7f099c0e66d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f09a4067b10 tx=0x7f09a406a260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a40ba650 con 0x7f099c0e3080 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f916dcb9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 0x7f916813a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f916dcb9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 0x7f916813a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57158/0 (socket says 172.21.15.111:57158) 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f916ff1d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9168164b40 0x7f9168160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f916ff1d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168139af0 con 0x7f9168164b40 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f916ff1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168139d60 con 0x7f916806a4c0 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f916ff1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9168160fb0 con 0x7f916806eea0 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f916d4b8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f9168150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f916dcb9640 1 -- 172.21.15.111:0/3796687651 learned_addr learned my addr 172.21.15.111:0/3796687651 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9573fff640 1 -- 172.21.15.111:0/1994314558 shutdown_connections 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9573fff640 1 --2- 172.21.15.111:0/1994314558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f95741471d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9573fff640 1 -- 172.21.15.111:0/1994314558 >> 172.21.15.111:0/1994314558 conn(0x7f957406a8b0 msgr2=0x7f957406acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9573fff640 1 -- 172.21.15.111:0/1994314558 shutdown_connections 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9573fff640 1 -- 172.21.15.111:0/1994314558 wait complete. 2023-12-04T05:39:22.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9573fff640 1 Processor -- start 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328800ce70 con 0x7f329015aa60 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3288011560 con 0x7f329015aa60 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288011910 con 0x7f329015aa60 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3290148b00 con 0x7f329015aa60 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329014a7f0 con 0x7f329015aa60 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7fc465edf640 1 Processor -- start 2023-12-04T05:39:22.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7fc465edf640 1 -- start start 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9f32c83640 1 Processor -- start 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.975+0000 7f9f32c83640 1 -- start start 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f32c83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 0x7f9f2c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f32c83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f2c0b38d0 0x7f9f2c1ff030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f32c83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f2c150280 0x7f9f2c15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f32c83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c146eb0 con 0x7f9f2c0afea0 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f32c83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c146230 con 0x7f9f2c150280 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f32c83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c146510 con 0x7f9f2c0b38d0 2023-12-04T05:39:22.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e71fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 0x7f93e015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e71fb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00aefb0 con 0x7f93e015b510 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e71fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00af290 con 0x7f93e0161a10 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e71fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00af570 con 0x7f93e0144b80 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e69fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 0x7f93e015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e69fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 0x7f93e015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57180/0 (socket says 172.21.15.111:57180) 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.976+0000 7f93e69fa640 1 -- 172.21.15.111:0/3589867071 learned_addr learned my addr 172.21.15.111:0/3589867071 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f93e69fa640 1 -- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0144b80 msgr2=0x7f93e00b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f93e69fa640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0144b80 0x7f93e00b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f93e69fa640 1 -- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93e015b510 msgr2=0x7f93e01ff3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f93e69fa640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93e015b510 0x7f93e01ff3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f93e69fa640 1 -- 172.21.15.111:0/3589867071 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d8009a70 con 0x7f93e0161a10 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c8962640 1 -- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 msgr2=0x7f88bc00c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c8962640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 0x7f88bc00c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f88c8962640 1 -- 172.21.15.111:0/3896281370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c4068870 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a40ba7b0 con 0x7f099c0e3080 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f30a1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 0x7f9f2c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f30a1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 0x7f9f2c153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58858/0 (socket says 172.21.15.111:58858) 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f30a1f640 1 -- 172.21.15.111:0/2810931320 learned_addr learned my addr 172.21.15.111:0/2810931320 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9f31220640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f2c150280 0x7f9f2c15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c052a40 con 0x7f099c0e3080 2023-12-04T05:39:22.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c052760 con 0x7f099c0e3080 2023-12-04T05:39:22.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f88c37fe640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 0x7f88bc00c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:22.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f93e69fa640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 0x7f93e015a3d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009eb0 tx=0x7f93d8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.977+0000 7f9573fff640 1 -- start start 2023-12-04T05:39:22.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc465edf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4600c4480 0x7fc4601468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc465edf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 0x7fc460147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f88c8962640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 0x7f88bc00ca60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88c4079e40 tx=0x7f88c406a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc465edf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc460161800 0x7fc46015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc465edf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46015bb20 con 0x7fc4600cdd20 2023-12-04T05:39:22.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc465edf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46015bdc0 con 0x7fc4600c4480 2023-12-04T05:39:22.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc465edf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46015c0a0 con 0x7fc460161800 2023-12-04T05:39:22.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d803f420 con 0x7f93e0161a10 2023-12-04T05:39:22.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f9573fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f957415b510 0x7f957413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a40bab20 con 0x7f099c0e3080 2023-12-04T05:39:22.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f9573fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9574161a10 0x7f95741429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc45ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 0x7fc460147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc45ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 0x7fc460147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58884/0 (socket says 172.21.15.111:58884) 2023-12-04T05:39:22.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f9573fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f9574142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc45ffff640 1 -- 172.21.15.111:0/2575727791 learned_addr learned my addr 172.21.15.111:0/2575727791 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7fc4656de640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc460161800 0x7fc46015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d803f580 con 0x7f93e0161a10 2023-12-04T05:39:22.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d803f910 con 0x7f93e0161a10 2023-12-04T05:39:22.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e00af850 con 0x7f93e0161a10 2023-12-04T05:39:22.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e0143dc0 con 0x7f93e0161a10 2023-12-04T05:39:22.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f916e4ba640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9168164b40 0x7f9168160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9573fff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95740b1900 con 0x7f9574165a10 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9573fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95740b0d80 con 0x7f957415b510 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9573fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95740b1060 con 0x7f9574161a10 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f9574142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f9574142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58900/0 (socket says 172.21.15.111:58900) 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 -- 172.21.15.111:0/3287790710 learned_addr learned my addr 172.21.15.111:0/3287790710 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f916dcb9640 1 -- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 msgr2=0x7f9168150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f916dcb9640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f9168150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f916dcb9640 1 -- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9168164b40 msgr2=0x7f9168160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f916dcb9640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9168164b40 0x7f9168160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f916dcb9640 1 -- 172.21.15.111:0/3796687651 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009b30 con 0x7f916806a4c0 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9572ffd640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f957415b510 0x7f957413d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95727fc640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9574161a10 0x7f95741429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7fc45ffff640 1 -- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc460161800 msgr2=0x7fc46015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7fc45ffff640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc460161800 0x7fc46015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7fc45ffff640 1 -- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4600c4480 msgr2=0x7fc4601468b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7fc45ffff640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4600c4480 0x7fc4601468b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7fc45ffff640 1 -- 172.21.15.111:0/2575727791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458009b30 con 0x7fc4600cdd20 2023-12-04T05:39:22.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f916dcb9640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 0x7f916813a520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f916000b3f0 tx=0x7f916000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9f30a1f640 1 -- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f2c0b38d0 msgr2=0x7f9f2c1ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9f30a1f640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f2c0b38d0 0x7f9f2c1ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9f30a1f640 1 -- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f2c150280 msgr2=0x7f9f2c15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9f30a1f640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f2c150280 0x7f9f2c15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f9f30a1f640 1 -- 172.21.15.111:0/2810931320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20009400 con 0x7f9f2c0afea0 2023-12-04T05:39:22.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 -- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9574161a10 msgr2=0x7f95741429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9574161a10 0x7f95741429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 -- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f957415b510 msgr2=0x7f957413d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f957415b510 0x7f957413d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7fc45ffff640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 0x7fc460147230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e50 tx=0x7fc458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95737fe640 1 -- 172.21.15.111:0/3287790710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c009b30 con 0x7f9574165a10 2023-12-04T05:39:22.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f95727fc640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9574161a10 0x7f95741429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:22.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0984005250 con 0x7f099c0e3080 2023-12-04T05:39:22.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.979+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40b8670 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011830 con 0x7f916806a4c0 2023-12-04T05:39:22.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9160011990 con 0x7f916806a4c0 2023-12-04T05:39:22.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3896281370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc00b4d0 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3896281370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc00bcd0 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c40a6cf0 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40b8a10 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f09a40bac80 con 0x7f099c0e3080 2023-12-04T05:39:22.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f09a17fa640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f098806b370 0x7f098806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f95737fe640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f9574142f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f956c036040 tx=0x7f956c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3288017070 con 0x7f329015aa60 2023-12-04T05:39:22.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.978+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3278005250 con 0x7f329015aa60 2023-12-04T05:39:22.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458007e70 con 0x7fc4600cdd20 2023-12-04T05:39:22.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45803e450 con 0x7fc4600cdd20 2023-12-04T05:39:22.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45803e800 con 0x7fc4600cdd20 2023-12-04T05:39:22.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f9f30a1f640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 0x7f9f2c153f40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400efc0 tx=0x7f9f2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f93d4ff9640 1 -- 172.21.15.111:0/3589867071 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc005250 con 0x7f93e0161a10 2023-12-04T05:39:22.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46015c380 con 0x7fc4600cdd20 2023-12-04T05:39:22.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc460146df0 con 0x7fc4600cdd20 2023-12-04T05:39:22.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.980+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f93d803fa70 con 0x7f93e0161a10 2023-12-04T05:39:22.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f93d6ffd640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f93c406b370 0x7f93c406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93d80ac670 con 0x7f93e0161a10 2023-12-04T05:39:22.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f32867fc640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f327406b3a0 0x7f327406d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c03e450 con 0x7f9574165a10 2023-12-04T05:39:22.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f328808df30 con 0x7f329015aa60 2023-12-04T05:39:22.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011d20 con 0x7f916806a4c0 2023-12-04T05:39:22.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91681609f0 con 0x7f916806a4c0 2023-12-04T05:39:22.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916813a000 con 0x7f916806a4c0 2023-12-04T05:39:22.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24014480 con 0x7f9f2c0afea0 2023-12-04T05:39:22.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c03e5d0 con 0x7f9574165a10 2023-12-04T05:39:22.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95740b1340 con 0x7f9574165a10 2023-12-04T05:39:22.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c1467f0 con 0x7f9f2c0afea0 2023-12-04T05:39:22.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3896281370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a4005250 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957413de00 con 0x7f9574165a10 2023-12-04T05:39:22.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c154480 con 0x7f9f2c0afea0 2023-12-04T05:39:22.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2400f040 con 0x7f9f2c0afea0 2023-12-04T05:39:22.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.981+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24014800 con 0x7f9f2c0afea0 2023-12-04T05:39:22.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c03e980 con 0x7f9574165a10 2023-12-04T05:39:22.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7f09a89d9640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f098806b370 0x7f098806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09a4126170 con 0x7f099c0e3080 2023-12-04T05:39:22.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7f3295ea8640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f327406b3a0 0x7f327406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7f93e61f9640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f93c406b370 0x7f93c406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46015c8a0 con 0x7fc4600cdd20 2023-12-04T05:39:22.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.983+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574069360 con 0x7f9574165a10 2023-12-04T05:39:22.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.982+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916813a220 con 0x7f916806a4c0 2023-12-04T05:39:22.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.983+0000 7f3295ea8640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f327406b3a0 0x7f327406d860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3280034000 tx=0x7f328002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.983+0000 7f93e61f9640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f93c406b370 0x7f93c406d830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0015f0 tx=0x7f93dc007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc45803e960 con 0x7fc4600cdd20 2023-12-04T05:39:22.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.983+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c069360 con 0x7f9f2c0afea0 2023-12-04T05:39:22.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7f09a89d9640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f098806b370 0x7f098806d830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0998006f40 tx=0x7f0998009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f956c02cd50 con 0x7f9574165a10 2023-12-04T05:39:22.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7fc45d7fa640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc44c06b2f0 0x7fc44c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4580ac6f0 con 0x7fc4600cdd20 2023-12-04T05:39:22.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f88c40b5040 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9f24014960 con 0x7f9f2c0afea0 2023-12-04T05:39:22.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7fc464edd640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc44c06b2f0 0x7fc44c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.984+0000 7f9f297fa640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f1806b2f0 0x7f9f1806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f88c0ff9640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f88a806b2f0 0x7f88a806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88c4126020 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f9f2bfff640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f1806b2f0 0x7f9f1806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f88c3fff640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f88a806b2f0 0x7f88a806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f955b7fe640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f956006b2a0 0x7f956006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f956c0ac7b0 con 0x7f9574165a10 2023-12-04T05:39:22.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f9572ffd640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f956006b2a0 0x7f956006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f9f2bfff640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f1806b2f0 0x7f9f1806d7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c150a60 tx=0x7f9f1c007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09a40f1120 con 0x7f099c0e3080 2023-12-04T05:39:22.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7f88c3fff640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f88a806b2f0 0x7f88a806d7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88b8006a90 tx=0x7f88b8006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.985+0000 7fc464edd640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc44c06b2f0 0x7fc44c06d7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc450000f60 tx=0x7fc4500079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.986+0000 7f9572ffd640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f956006b2a0 0x7f956006d760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9564006fd0 tx=0x7f95640090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.986+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3288058e10 con 0x7f329015aa60 2023-12-04T05:39:22.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.988+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88c40f0f70 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.989+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f2408d070 con 0x7f9f2c0afea0 2023-12-04T05:39:22.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.990+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93d8077670 con 0x7f93e0161a10 2023-12-04T05:39:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.991+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9160004020 con 0x7f916806a4c0 2023-12-04T05:39:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.992+0000 7f915e7fc640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f915406b2f0 0x7f915406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.992+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9160089ca0 con 0x7f916806a4c0 2023-12-04T05:39:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.992+0000 7f916d4b8640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f915406b2f0 0x7f915406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:22.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.993+0000 7f916d4b8640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f915406b2f0 0x7f915406d7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f916816d510 tx=0x7f91640090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:22.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.996+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc46015c8a0 con 0x7fc4600cdd20 2023-12-04T05:39:22.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:21.996+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9160089e60 con 0x7f916806a4c0 2023-12-04T05:39:22.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.002+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f956c077610 con 0x7f9574165a10 2023-12-04T05:39:22.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.010+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f24055650 con 0x7f9f2c0afea0 2023-12-04T05:39:22.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.374+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0984005500 con 0x7f099c0e3080 2023-12-04T05:39:22.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.375+0000 7f09a17fa640 1 -- 172.21.15.111:0/2368140209 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09a40f4740 con 0x7f099c0e3080 2023-12-04T05:39:22.376 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:22.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.376+0000 7f93d4ff9640 1 -- 172.21.15.111:0/3589867071 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93bc005500 con 0x7f93e0161a10 2023-12-04T05:39:22.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.377+0000 7f93d6ffd640 1 -- 172.21.15.111:0/3589867071 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93d807ac90 con 0x7f93e0161a10 2023-12-04T05:39:22.378 INFO:teuthology.orchestra.run.smithi111.stdout:55834574854 2023-12-04T05:39:22.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.378+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f098806b370 msgr2=0x7f098806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.378+0000 7f09a99db640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f098806b370 0x7f098806d830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0998006f40 tx=0x7f0998009240 comp rx=0 tx=0).stop 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c0e3080 msgr2=0x7f099c0e66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c0e3080 0x7f099c0e66d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f09a4067b10 tx=0x7f09a406a260 comp rx=0 tx=0).stop 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 shutdown_connections 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f098806b370 0x7f098806d830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c0e3080 0x7f099c0e66d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c0540f0 0x7f099c051a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 --2- 172.21.15.111:0/2368140209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f099c03e800 0x7f099c188bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.379+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 >> 172.21.15.111:0/2368140209 conn(0x7f099c0559a0 msgr2=0x7f099c0dd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f93c406b370 msgr2=0x7f93c406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f93c406b370 0x7f93c406d830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0015f0 tx=0x7f93dc007a40 comp rx=0 tx=0).stop 2023-12-04T05:39:22.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 msgr2=0x7f93e015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 0x7f93e015a3d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009eb0 tx=0x7f93d8004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:22.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 shutdown_connections 2023-12-04T05:39:22.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 shutdown_connections 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f93c406b370 0x7f93c406d830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0161a10 0x7f93e015a3d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93e015b510 0x7f93e01ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 --2- 172.21.15.111:0/3589867071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93e0144b80 0x7f93e00b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 >> 172.21.15.111:0/3589867071 conn(0x7f93e006a8b0 msgr2=0x7f93e0163ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f09a99db640 1 -- 172.21.15.111:0/2368140209 wait complete. 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 shutdown_connections 2023-12-04T05:39:22.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.380+0000 7f93e71fb640 1 -- 172.21.15.111:0/3589867071 wait complete. 2023-12-04T05:39:22.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.387+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9168067460 con 0x7f916806a4c0 2023-12-04T05:39:22.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.388+0000 7f915e7fc640 1 -- 172.21.15.111:0/3796687651 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9160058630 con 0x7f916806a4c0 2023-12-04T05:39:22.389 INFO:teuthology.orchestra.run.smithi111.stdout:55834574854 2023-12-04T05:39:22.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f915406b2f0 msgr2=0x7f915406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f915406b2f0 0x7f915406d7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f916816d510 tx=0x7f91640090e0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 msgr2=0x7f916813a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.392+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9574069360 con 0x7f9574165a10 2023-12-04T05:39:22.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 0x7f916813a520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f916000b3f0 tx=0x7f916000b8b0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 shutdown_connections 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f915406b2f0 0x7f915406d7b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9168164b40 0x7f9168160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f916806eea0 0x7f9168150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 --2- 172.21.15.111:0/3796687651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f916806a4c0 0x7f916813a520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.391+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 >> 172.21.15.111:0/3796687651 conn(0x7f91680cfec0 msgr2=0x7f916814d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.392+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 shutdown_connections 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.392+0000 7f916ff1d640 1 -- 172.21.15.111:0/3796687651 wait complete. 2023-12-04T05:39:22.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.392+0000 7f955b7fe640 1 -- 172.21.15.111:0/3287790710 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f956c07ac30 con 0x7f9574165a10 2023-12-04T05:39:22.397 INFO:teuthology.orchestra.run.smithi111.stdout:55834574854 2023-12-04T05:39:22.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.393+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3278005500 con 0x7f329015aa60 2023-12-04T05:39:22.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.394+0000 7f32867fc640 1 -- 172.21.15.111:0/165143087 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f328805c430 con 0x7f329015aa60 2023-12-04T05:39:22.398 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:22.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.394+0000 7f88ca3c5640 1 -- 172.21.15.111:0/3896281370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f88a40057d0 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.395+0000 7f88c0ff9640 1 -- 172.21.15.111:0/3896281370 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88c40bf090 con 0x7f88bc0eb2c0 2023-12-04T05:39:22.399 INFO:teuthology.orchestra.run.smithi111.stdout:55834574854 2023-12-04T05:39:22.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.395+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f956006b2a0 msgr2=0x7f956006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.395+0000 7f9573fff640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f956006b2a0 0x7f956006d760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9564006fd0 tx=0x7f95640090e0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.395+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 msgr2=0x7f9574142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.395+0000 7f9573fff640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f9574142f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f956c036040 tx=0x7f956c004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:22.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.396+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 shutdown_connections 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.396+0000 7f9573fff640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f956006b2a0 0x7f956006d760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.396+0000 7f9573fff640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9574165a10 0x7f9574142f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.396+0000 7f9573fff640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9574161a10 0x7f95741429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.396+0000 7f9573fff640 1 --2- 172.21.15.111:0/3287790710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f957415b510 0x7f957413d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.396+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 >> 172.21.15.111:0/3287790710 conn(0x7f957406a8b0 msgr2=0x7f9574147510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.397+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 shutdown_connections 2023-12-04T05:39:22.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.397+0000 7f9573fff640 1 -- 172.21.15.111:0/3287790710 wait complete. 2023-12-04T05:39:22.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-12-04T05:39:22.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f327406b3a0 msgr2=0x7f327406d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f327406b3a0 0x7f327406d860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3280034000 tx=0x7f328002f000 comp rx=0 tx=0).stop 2023-12-04T05:39:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f329015aa60 msgr2=0x7f329014e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 -- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f88a806b2f0 msgr2=0x7f88a806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f88a806b2f0 0x7f88a806d7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88b8006a90 tx=0x7f88b8006a20 comp rx=0 tx=0).stop 2023-12-04T05:39:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f329015aa60 0x7f329014e7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3288001700 tx=0x7f328800a430 comp rx=0 tx=0).stop 2023-12-04T05:39:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 shutdown_connections 2023-12-04T05:39:22.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 -- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 msgr2=0x7f88bc00ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 0x7f88bc00ca60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88c4079e40 tx=0x7f88c406a260 comp rx=0 tx=0).stop 2023-12-04T05:39:22.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f327406b3a0 0x7f327406d860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f329015aa60 0x7f329014e7c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32900aed80 0x7f3290144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 --2- 172.21.15.111:0/165143087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900662b0 0x7f329014bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 >> 172.21.15.111:0/165143087 conn(0x7f32900cf600 msgr2=0x7f32900cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 -- 172.21.15.111:0/3896281370 shutdown_connections 2023-12-04T05:39:22.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f88a806b2f0 0x7f88a806d7b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88bc0eb2c0 0x7f88bc00ca60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88bc0e77e0 0x7f88bc00c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 --2- 172.21.15.111:0/3896281370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88bc006160 0x7f88bc04f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f88ae7fc640 1 -- 172.21.15.111:0/3896281370 >> 172.21.15.111:0/3896281370 conn(0x7f88bc0559a0 msgr2=0x7f88bc038a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 shutdown_connections 2023-12-04T05:39:22.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.398+0000 7f329810c640 1 -- 172.21.15.111:0/165143087 wait complete. 2023-12-04T05:39:22.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.399+0000 7f88ae7fc640 1 -- 172.21.15.111:0/3896281370 shutdown_connections 2023-12-04T05:39:22.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.399+0000 7f88ae7fc640 1 -- 172.21.15.111:0/3896281370 wait complete. 2023-12-04T05:39:22.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.400+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc46015c1e0 con 0x7fc4600cdd20 2023-12-04T05:39:22.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.401+0000 7fc45d7fa640 1 -- 172.21.15.111:0/2575727791 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc46015c1e0 con 0x7fc4600cdd20 2023-12-04T05:39:22.407 INFO:teuthology.orchestra.run.smithi111.stdout:55834574854 2023-12-04T05:39:22.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.404+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc44c06b2f0 msgr2=0x7fc44c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.404+0000 7fc465edf640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc44c06b2f0 0x7fc44c06d7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc450000f60 tx=0x7fc4500079b0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.404+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 msgr2=0x7fc460147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.404+0000 7fc465edf640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 0x7fc460147230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e50 tx=0x7fc458004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 shutdown_connections 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc44c06b2f0 0x7fc44c06d7b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc460161800 0x7fc46015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4600cdd20 0x7fc460147230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 --2- 172.21.15.111:0/2575727791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4600c4480 0x7fc4601468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 >> 172.21.15.111:0/2575727791 conn(0x7fc4600d0da0 msgr2=0x7fc46006e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 shutdown_connections 2023-12-04T05:39:22.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.405+0000 7fc465edf640 1 -- 172.21.15.111:0/2575727791 wait complete. 2023-12-04T05:39:22.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.408+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9f2c0b32d0 con 0x7f9f2c0afea0 2023-12-04T05:39:22.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.408+0000 7f9f297fa640 1 -- 172.21.15.111:0/2810931320 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f24058c70 con 0x7f9f2c0afea0 2023-12-04T05:39:22.410 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:22.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.411+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f1806b2f0 msgr2=0x7f9f1806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.411+0000 7f9f32c83640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f1806b2f0 0x7f9f1806d7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c150a60 tx=0x7f9f1c007a40 comp rx=0 tx=0).stop 2023-12-04T05:39:22.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.411+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 msgr2=0x7f9f2c153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.411+0000 7f9f32c83640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 0x7f9f2c153f40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400efc0 tx=0x7f9f2400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 shutdown_connections 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f1806b2f0 0x7f9f1806d7b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f2c150280 0x7f9f2c15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f2c0b38d0 0x7f9f2c1ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 --2- 172.21.15.111:0/2810931320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f2c0afea0 0x7f9f2c153f40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 >> 172.21.15.111:0/2810931320 conn(0x7f9f2c0aea80 msgr2=0x7f9f2c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:22.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-12-04T05:39:22.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 shutdown_connections 2023-12-04T05:39:22.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:22.412+0000 7f9f32c83640 1 -- 172.21.15.111:0/2810931320 wait complete. 2023-12-04T05:39:22.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-12-04T05:39:22.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-12-04T05:39:22.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-12-04T05:39:22.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-12-04T05:39:22.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-12-04T05:39:23.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-04T05:39:23.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-04T05:39:23.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-04T05:39:23.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-04T05:39:23.420 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-04T05:39:23.420 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-04T05:39:23.426 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-04T05:39:23.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-04T05:39:23.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.592+0000 7fd8e086a640 1 Processor -- start 2023-12-04T05:39:23.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.592+0000 7fd8e086a640 1 -- start start 2023-12-04T05:39:23.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.592+0000 7f26b5b05640 1 Processor -- start 2023-12-04T05:39:23.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.593+0000 7f26b5b05640 1 -- start start 2023-12-04T05:39:23.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.593+0000 7eff9655c640 1 Processor -- start 2023-12-04T05:39:23.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7fd8e086a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d404f8d0 0x7fd8d404fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7fd8e086a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8d4053790 0x7fd8d4007130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7fd8e086a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d40548d0 con 0x7fd8d4005e60 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7fd8e086a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d40501f0 con 0x7fd8d4053790 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7fd8e086a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4037bf0 con 0x7fd8d404f8d0 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7f26b5b05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 0x7f26a8053220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7f26af7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f26a804fa50 0x7f26a804fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58728/0 (socket says 172.21.15.111:58728) 2023-12-04T05:39:23.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.593+0000 7eff9655c640 1 -- start start 2023-12-04T05:39:23.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7f0eb759e640 1 Processor -- start 2023-12-04T05:39:23.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7f26af7fe640 1 -- 172.21.15.111:0/1424786719 learned_addr learned my addr 172.21.15.111:0/1424786719 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:23.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.594+0000 7f26aeffd640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 0x7f26a8053220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a8006860 con 0x7f26a804fa50 2023-12-04T05:39:23.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a8006ab0 con 0x7f26a8039290 2023-12-04T05:39:23.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7f0eb759e640 1 -- start start 2023-12-04T05:39:23.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8007730 con 0x7f26a80df3e0 2023-12-04T05:39:23.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9655c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff880e3090 0x7eff88057d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9655c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7eff880542d0 0x7eff880068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9655c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff880582a0 con 0x7eff8803e810 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9655c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff88006170 con 0x7eff880542d0 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9655c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff88058540 con 0x7eff880e3090 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9555a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff880e3090 0x7eff88057d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9555a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff880e3090 0x7eff88057d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57230/0 (socket says 172.21.15.111:57230) 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff9555a640 1 -- 172.21.15.111:0/3507413367 learned_addr learned my addr 172.21.15.111:0/3507413367 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220953174 0 0) 0x7f26a8006ab0 con 0x7f26a8039290 2023-12-04T05:39:23.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7fdf5c961640 1 Processor -- start 2023-12-04T05:39:23.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26a0003520 con 0x7f26a8039290 2023-12-04T05:39:23.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7fd8dad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d404f8d0 0x7fd8d404fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7fd8da59c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8d4053790 0x7fd8d4007130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7fd8da59c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8d4053790 0x7fd8d4007130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57228/0 (socket says 172.21.15.111:57228) 2023-12-04T05:39:23.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7fd8da59c640 1 -- 172.21.15.111:0/2824547645 learned_addr learned my addr 172.21.15.111:0/2824547645 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7eff95d5b640 1 --2- 172.21.15.111:0/3507413367 >> v2:172.21.15.111:3300/0 conn(0x7eff880542d0 0x7eff880068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.595+0000 7fdf5c961640 1 -- start start 2023-12-04T05:39:23.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363394246 0 0) 0x7f26a0003520 con 0x7f26a8039290 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 msgr2=0x7f26a8053220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26ae7fc640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 0x7f26a8053220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 >> v1:172.21.15.111:6789/0 conn(0x7f26a804fa50 legacy=0x7f26a804fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8006d90 con 0x7f26a8039290 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f26b0068870 con 0x7f26a8039290 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f26aeffd640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 0x7f26a8053220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7eff9555a640 1 -- 172.21.15.111:0/3507413367 >> v2:172.21.15.111:3300/0 conn(0x7eff880542d0 msgr2=0x7eff880068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7eff9555a640 1 --2- 172.21.15.111:0/3507413367 >> v2:172.21.15.111:3300/0 conn(0x7eff880542d0 0x7eff880068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7eff9555a640 1 -- 172.21.15.111:0/3507413367 >> v1:172.21.15.111:6789/0 conn(0x7eff8803e810 legacy=0x7eff88053bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f0eb759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0eb80cde80 0x7f0eb80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f0eb759e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80d1d30 con 0x7f0eb80cfac0 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f0eb759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80ced50 con 0x7f0eb815e900 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f0eb759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb80ceff0 con 0x7f0eb80cde80 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f0eb659c640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f0eb815e900 0x7f0eb80cf3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43144/0 (socket says 172.21.15.111:43144) 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7f0eb659c640 1 -- 172.21.15.111:0/1204938648 learned_addr learned my addr 172.21.15.111:0/1204938648 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:23.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7eff9555a640 1 -- 172.21.15.111:0/3507413367 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff88058820 con 0x7eff880e3090 2023-12-04T05:39:23.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fe611edf640 1 Processor -- start 2023-12-04T05:39:23.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fe611edf640 1 -- start start 2023-12-04T05:39:23.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5c961640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf580b2610 0x7fdf580b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617633909 0 0) 0x7f0eb80d1d30 con 0x7f0eb80cfac0 2023-12-04T05:39:23.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7eff9555a640 1 --2- 172.21.15.111:0/3507413367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff880e3090 0x7eff88057d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff900692c0 tx=0x7eff900ae620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104fbeeb05bfc350 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 >> v1:172.21.15.192:6789/0 conn(0x7f26a8039290 legacy=0x7f26a8053760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea0003520 con 0x7f0eb80cfac0 2023-12-04T05:39:23.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f26ae7fc640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 0x7f26a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5c961640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580b2f30 con 0x7fdf5806a950 2023-12-04T05:39:23.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f26ae7fc640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a0003f60 0x7f26a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7eff87fff640 1 -- 172.21.15.111:0/3507413367 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900aeea0 con 0x7eff880e3090 2023-12-04T05:39:23.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5c961640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5806aea0 con 0x7fdf5806a4c0 2023-12-04T05:39:23.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5c961640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5806b180 con 0x7fdf580b2610 2023-12-04T05:39:23.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7eff87fff640 1 -- 172.21.15.111:0/3507413367 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff900b1050 con 0x7eff880e3090 2023-12-04T05:39:23.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f26ae7fc640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7eff87fff640 1 -- 172.21.15.111:0/3507413367 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900aeea0 con 0x7eff880e3090 2023-12-04T05:39:23.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5659c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf580b2610 0x7fdf580b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8007730 con 0x7f26a0003f60 2023-12-04T05:39:23.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5659c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf580b2610 0x7fdf580b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58920/0 (socket says 172.21.15.111:58920) 2023-12-04T05:39:23.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.597+0000 7fdf5659c640 1 -- 172.21.15.111:0/1430186125 learned_addr learned my addr 172.21.15.111:0/1430186125 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760228857 0 0) 0x7f0ea0003520 con 0x7f0eb80cfac0 2023-12-04T05:39:23.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 >> v2:172.21.15.192:3300/0 conn(0x7f0eb80cde80 msgr2=0x7f0eb80ae7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-04T05:39:23.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 --2- 172.21.15.111:0/1204938648 >> v2:172.21.15.192:3300/0 conn(0x7f0eb80cde80 0x7f0eb80ae7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 >> v1:172.21.15.192:6789/0 conn(0x7f0eb815e900 legacy=0x7f0eb80cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26aeffd640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a0003f60 0x7f26a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26affff640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a000b9b0 con 0x7f26a0004d00 2023-12-04T05:39:23.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80d1a90 con 0x7f0eb80cfac0 2023-12-04T05:39:23.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a000bb70 con 0x7f26a80df3e0 2023-12-04T05:39:23.611 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f0ea8001670 con 0x7f0eb80cfac0 2023-12-04T05:39:23.611 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7fd8da59c640 1 -- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d404f8d0 msgr2=0x7fd8d404fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.611 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7fd8da59c640 1 --2- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d404f8d0 0x7fd8d404fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.611 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7fd8da59c640 1 -- 172.21.15.111:0/2824547645 >> v1:172.21.15.192:6789/0 conn(0x7fd8d4005e60 legacy=0x7fd8d40062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.611 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.596+0000 7fd8da59c640 1 -- 172.21.15.111:0/2824547645 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d4053d50 con 0x7fd8d4053790 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe611edf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe60c06bbe0 0x7fe60c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe611edf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe60c160cc0 0x7fe60c0adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 >> v1:172.21.15.111:6789/0 conn(0x7f0eb80cfac0 legacy=0x7f0eb80ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00039c0 0x7f0ea0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ea00047c0 0x7f0ea0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea0005330 0x7f0ea000b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.612 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80ceff0 con 0x7f0ea00047c0 2023-12-04T05:39:23.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe610edd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe60c15ac00 0x7fe60c06b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58770/0 (socket says 172.21.15.111:58770) 2023-12-04T05:39:23.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe610edd640 1 -- 172.21.15.111:0/1793834903 learned_addr learned my addr 172.21.15.111:0/1793834903 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:23.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea000c280 con 0x7f0ea0005330 2023-12-04T05:39:23.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea000c440 con 0x7f0ea00039c0 2023-12-04T05:39:23.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60c15dd20 con 0x7fe60c15ac00 2023-12-04T05:39:23.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7eff9655c640 1 -- 172.21.15.111:0/3507413367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff880e3090 msgr2=0x7eff88057d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60c0ad800 con 0x7fe60c06bbe0 2023-12-04T05:39:23.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60c15e000 con 0x7fe60c160cc0 2023-12-04T05:39:23.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7f0eb5d9b640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ea00047c0 0x7f0ea0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7eff9655c640 1 --2- 172.21.15.111:0/3507413367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff880e3090 0x7eff88057d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff900692c0 tx=0x7eff900ae620 comp rx=0 tx=0).stop 2023-12-04T05:39:23.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7fe60b7fe640 1 -- 172.21.15.111:0/1793834903 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814200459 0 0) 0x7fe60c15dd20 con 0x7fe60c15ac00 2023-12-04T05:39:23.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7eff9655c640 1 -- 172.21.15.111:0/3507413367 shutdown_connections 2023-12-04T05:39:23.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7eff9655c640 1 --2- 172.21.15.111:0/3507413367 >> v2:172.21.15.111:3300/0 conn(0x7eff880542d0 0x7eff880068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7eff9655c640 1 --2- 172.21.15.111:0/3507413367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff880e3090 0x7eff88057d60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7eff9655c640 1 -- 172.21.15.111:0/3507413367 >> 172.21.15.111:0/3507413367 conn(0x7eff880559b0 msgr2=0x7eff88055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7fe60b7fe640 1 -- 172.21.15.111:0/1793834903 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5fc003520 con 0x7fe60c15ac00 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26affff640 1 -- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 msgr2=0x7f26a0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26affff640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a80df3e0 0x7f26a0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26affff640 1 -- 172.21.15.111:0/1424786719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a0003f60 msgr2=0x7f26a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26affff640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a0003f60 0x7f26a00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7f26affff640 1 -- 172.21.15.111:0/1424786719 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0003520 con 0x7f26a0004d00 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7fe60bfff640 1 --2- 172.21.15.111:0/1793834903 >> v2:172.21.15.111:3300/0 conn(0x7fe60c06bbe0 0x7fe60c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7eff9655c640 1 -- 172.21.15.111:0/3507413367 shutdown_connections 2023-12-04T05:39:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7f0eb5d9b640 1 -- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00039c0 msgr2=0x7f0ea0003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:39:23.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7eff9655c640 1 -- 172.21.15.111:0/3507413367 wait complete. 2023-12-04T05:39:23.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fdf5659c640 1 -- 172.21.15.111:0/1430186125 >> v1:172.21.15.192:6789/0 conn(0x7fdf5806a4c0 legacy=0x7fdf580ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fdf5659c640 1 -- 172.21.15.111:0/1430186125 >> v1:172.21.15.111:6789/0 conn(0x7fdf5806a950 legacy=0x7fdf5816b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.598+0000 7fdf5659c640 1 -- 172.21.15.111:0/1430186125 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5806b460 con 0x7fdf580b2610 2023-12-04T05:39:23.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7fd8da59c640 1 --2- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4053790 0x7fd8d4007130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc02ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3b401d3af375e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7fdf5659c640 1 --2- 172.21.15.111:0/1430186125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf580b2610 0x7fdf580b29f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf5002bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d2571a4f622576 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7f26affff640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a0005180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26b004faf0 tx=0x7f26a00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7825ab6601cb7be server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fdf5559a640 1 -- 172.21.15.111:0/1430186125 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5002cb30 con 0x7fdf580b2610 2023-12-04T05:39:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7fd8d9d9b640 1 -- 172.21.15.111:0/2824547645 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc02f850 con 0x7fd8d4053790 2023-12-04T05:39:23.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fe60bfff640 1 -- 172.21.15.111:0/1793834903 >> v2:172.21.15.192:3300/0 conn(0x7fe60c160cc0 msgr2=0x7fe60c0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fdf5559a640 1 -- 172.21.15.111:0/1430186125 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5002cc90 con 0x7fdf580b2610 2023-12-04T05:39:23.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fd8d9d9b640 1 -- 172.21.15.111:0/2824547645 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc02f9b0 con 0x7fd8d4053790 2023-12-04T05:39:23.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fe60bfff640 1 --2- 172.21.15.111:0/1793834903 >> v2:172.21.15.192:3300/0 conn(0x7fe60c160cc0 0x7fe60c0adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fe60bfff640 1 -- 172.21.15.111:0/1793834903 >> v1:172.21.15.111:6789/0 conn(0x7fe60c15ac00 legacy=0x7fe60c06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b00a8c70 con 0x7f26a0004d00 2023-12-04T05:39:23.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00a8df0 con 0x7f26a0004d00 2023-12-04T05:39:23.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fe60bfff640 1 -- 172.21.15.111:0/1793834903 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c15e2e0 con 0x7fe60c06bbe0 2023-12-04T05:39:23.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f26ae7fc640 1 -- 172.21.15.111:0/1424786719 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b00b0680 con 0x7f26a0004d00 2023-12-04T05:39:23.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 -- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4053790 msgr2=0x7fd8d4007130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 --2- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4053790 0x7fd8d4007130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc02ba30 comp rx=0 tx=0).stop 2023-12-04T05:39:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7fe60b7fe640 1 -- 172.21.15.111:0/1793834903 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1528740295 0 0) 0x7fe5fc003520 con 0x7fe60c15ac00 2023-12-04T05:39:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 msgr2=0x7f26a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 -- 172.21.15.111:0/1430186125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf580b2610 msgr2=0x7fdf580b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 --2- 172.21.15.111:0/1430186125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf580b2610 0x7fdf580b29f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf5002bf90 comp rx=0 tx=0).stop 2023-12-04T05:39:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26b5b05640 1 --2- 172.21.15.111:0/1424786719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a0005180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26b004faf0 tx=0x7f26a00057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 -- 172.21.15.111:0/1430186125 shutdown_connections 2023-12-04T05:39:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 --2- 172.21.15.111:0/1430186125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf580b2610 0x7fdf580b29f0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf5002bf90 comp rx=0 tx=0).stop 2023-12-04T05:39:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 -- 172.21.15.111:0/1430186125 >> 172.21.15.111:0/1430186125 conn(0x7fdf580b1310 msgr2=0x7fdf580b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26affff640 1 -- 172.21.15.111:0/1424786719 reap_dead start 2023-12-04T05:39:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.599+0000 7f0eb6d9d640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea0005330 0x7f0ea000b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f0eb5d9b640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00039c0 0x7f0ea0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f0eb5d9b640 1 -- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea0005330 msgr2=0x7f0ea000b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f0eb5d9b640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea0005330 0x7f0ea000b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.600+0000 7f0eb5d9b640 1 -- 172.21.15.111:0/1204938648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0003520 con 0x7f0ea00047c0 2023-12-04T05:39:23.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7eff9655c640 1 Processor -- start 2023-12-04T05:39:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 -- 172.21.15.111:0/2824547645 shutdown_connections 2023-12-04T05:39:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 --2- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4053790 0x7fd8d4007130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f0eb6d9d640 1 -- 172.21.15.111:0/1204938648 reap_dead start 2023-12-04T05:39:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 --2- 172.21.15.111:0/2824547645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d404f8d0 0x7fd8d404fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 -- 172.21.15.111:0/2824547645 >> 172.21.15.111:0/2824547645 conn(0x7fd8d4009f80 msgr2=0x7fd8d400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 shutdown_connections 2023-12-04T05:39:23.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 >> 172.21.15.111:0/1424786719 conn(0x7f26a8009f70 msgr2=0x7f26a800a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fd8e086a640 1 -- 172.21.15.111:0/2824547645 shutdown_connections 2023-12-04T05:39:23.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7eff9655c640 1 -- start start 2023-12-04T05:39:23.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 -- 172.21.15.111:0/1430186125 shutdown_connections 2023-12-04T05:39:23.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fdf5c961640 1 -- 172.21.15.111:0/1430186125 wait complete. 2023-12-04T05:39:23.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f0eb5d9b640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ea00047c0 0x7f0ea0004c40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8007f40 tx=0x7f0ea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e459fd078e81b709 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 shutdown_connections 2023-12-04T05:39:23.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7fe60bfff640 1 --2- 172.21.15.111:0/1793834903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c06bbe0 0x7fe60c15d6d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097a0 tx=0x7fe5f802b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d3a084d050cc76 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea802c970 con 0x7f0ea00047c0 2023-12-04T05:39:23.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f26b5b05640 1 -- 172.21.15.111:0/1424786719 wait complete. 2023-12-04T05:39:23.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.601+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea802cad0 con 0x7f0ea00047c0 2023-12-04T05:39:23.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7fe60b7fe640 1 -- 172.21.15.111:0/1793834903 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600001670 con 0x7fe60c06bbe0 2023-12-04T05:39:23.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7fe60b7fe640 1 -- 172.21.15.111:0/1793834903 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f802e050 con 0x7fe60c06bbe0 2023-12-04T05:39:23.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7fe60b7fe640 1 -- 172.21.15.111:0/1793834903 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600001670 con 0x7fe60c06bbe0 2023-12-04T05:39:23.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7f0eb559a640 1 -- 172.21.15.111:0/1204938648 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea802c970 con 0x7f0ea00047c0 2023-12-04T05:39:23.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7f0eb759e640 1 -- 172.21.15.111:0/1204938648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ea00047c0 msgr2=0x7f0ea0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ea00047c0 0x7f0ea0004c40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8007f40 tx=0x7f0ea00057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c06bbe0 msgr2=0x7fe60c15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 --2- 172.21.15.111:0/1793834903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c06bbe0 0x7fe60c15d6d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097a0 tx=0x7fe5f802b730 comp rx=0 tx=0).stop 2023-12-04T05:39:23.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7f0eb759e640 1 -- 172.21.15.111:0/1204938648 shutdown_connections 2023-12-04T05:39:23.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1204938648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ea00047c0 0x7f0ea0004c40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7f0eb759e640 1 -- 172.21.15.111:0/1204938648 >> 172.21.15.111:0/1204938648 conn(0x7f0eb80d0dd0 msgr2=0x7f0eb80d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 shutdown_connections 2023-12-04T05:39:23.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.602+0000 7fd8e086a640 1 -- 172.21.15.111:0/2824547645 wait complete. 2023-12-04T05:39:23.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7f0eb759e640 1 -- 172.21.15.111:0/1204938648 shutdown_connections 2023-12-04T05:39:23.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 --2- 172.21.15.111:0/1793834903 >> v2:172.21.15.192:3300/0 conn(0x7fe60c160cc0 0x7fe60c0adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 --2- 172.21.15.111:0/1793834903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c06bbe0 0x7fe60c15d6d0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097a0 tx=0x7fe5f802b730 comp rx=0 tx=0).stop 2023-12-04T05:39:23.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 >> 172.21.15.111:0/1793834903 conn(0x7fe60c070a80 msgr2=0x7fe60c070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7eff9655c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff8803e810 0x7eff880d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7f26b5b05640 1 Processor -- start 2023-12-04T05:39:23.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 shutdown_connections 2023-12-04T05:39:23.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7fe611edf640 1 -- 172.21.15.111:0/1793834903 wait complete. 2023-12-04T05:39:23.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7eff9655c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff880542d0 0x7eff8803d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7eff9655c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 0x7eff8803dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7f26b5b05640 1 -- start start 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7eff95d5b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 0x7eff8803dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7eff95d5b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 0x7eff8803dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58950/0 (socket says 172.21.15.111:58950) 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7eff95d5b640 1 -- 172.21.15.111:0/4147956536 learned_addr learned my addr 172.21.15.111:0/4147956536 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88052430 con 0x7eff880e3090 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880517f0 con 0x7eff8803e810 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff88051a90 con 0x7eff880542d0 2023-12-04T05:39:23.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7fd8e086a640 1 Processor -- start 2023-12-04T05:39:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.603+0000 7f0eb759e640 1 -- 172.21.15.111:0/1204938648 wait complete. 2023-12-04T05:39:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7f0eb759e640 1 Processor -- start 2023-12-04T05:39:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fd8e086a640 1 -- start start 2023-12-04T05:39:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7f26b5b05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a80de510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7eff95d5b640 1 -- 172.21.15.111:0/4147956536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff880542d0 msgr2=0x7eff8803d7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-04T05:39:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7f26b5b05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 0x7f26a804d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7eff95d5b640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff880542d0 0x7eff8803d7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7f26b5b05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a804de60 0x7f26a8053410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fe611edf640 1 Processor -- start 2023-12-04T05:39:23.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7eff95d5b640 1 -- 172.21.15.111:0/4147956536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff8803e810 msgr2=0x7eff880d62d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-04T05:39:23.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7eff95d5b640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff8803e810 0x7eff880d62d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7eff95d5b640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9004f770 con 0x7eff880e3090 2023-12-04T05:39:23.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fe611edf640 1 -- start start 2023-12-04T05:39:23.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7f26aeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 0x7f26a804d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7fdf5c961640 1 Processor -- start 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.604+0000 7fdf5c961640 1 -- start start 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fdf5c961640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf5806a4c0 0x7fdf581572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fdf5c961640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5806a950 0x7fdf581642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fdf5c961640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 0x7fdf5815db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fdf5c961640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581ece70 con 0x7fdf5806a950 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fdf5c961640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58164cd0 con 0x7fdf5806a4c0 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7fdf5c961640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58157820 con 0x7fdf58164820 2023-12-04T05:39:23.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f26aeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 0x7f26a804d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58962/0 (socket says 172.21.15.111:58962) 2023-12-04T05:39:23.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7eff95d5b640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 0x7eff8803dcf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff8000eab0 tx=0x7eff8000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f26aeffd640 1 -- 172.21.15.111:0/768774721 learned_addr learned my addr 172.21.15.111:0/768774721 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f26b5b05640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a804e3c0 con 0x7f26a817c1a0 2023-12-04T05:39:23.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f26b5b05640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a804e6a0 con 0x7f26a0004d00 2023-12-04T05:39:23.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f26b5b05640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80dda30 con 0x7f26a804de60 2023-12-04T05:39:23.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f26affff640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a804de60 0x7f26a8053410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8e086a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4005e60 0x7fd8d4049c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000c910 con 0x7eff880e3090 2023-12-04T05:39:23.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8e086a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 0x7fd8d40d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8e086a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d4053790 0x7fd8d40e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8e086a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40e38a0 con 0x7fd8d404f8d0 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8e086a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40e2940 con 0x7fd8d4005e60 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8e086a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40e2c20 con 0x7fd8d4053790 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.605+0000 7f0eb759e640 1 -- start start 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fdf56d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 0x7fdf5815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fdf56d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 0x7fdf5815db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40252/0 (socket says 172.21.15.111:40252) 2023-12-04T05:39:23.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fdf56d9d640 1 -- 172.21.15.111:0/3727883122 learned_addr learned my addr 172.21.15.111:0/3727883122 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fdf55d9b640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5806a950 0x7fdf581642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fe611edf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 0x7fe60c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fd8db59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d4053790 0x7fd8d40e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 0x7f0eb81f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb816c830 0x7f0eb816b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb759e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb80b0580 0x7f0eb80b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb759e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb816b9c0 con 0x7f0eb80b0580 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81f3fe0 con 0x7f0ea00047c0 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81f3180 con 0x7f0eb816c830 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fe611edf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60c160cc0 0x7fe60c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 0x7f0eb81f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 0x7f0eb81f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57282/0 (socket says 172.21.15.111:57282) 2023-12-04T05:39:23.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7f0eb659c640 1 -- 172.21.15.111:0/1699163168 learned_addr learned my addr 172.21.15.111:0/1699163168 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7fdf5659c640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf5806a4c0 0x7fdf581572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fdf56d9d640 1 -- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf5806a4c0 msgr2=0x7fdf581572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26aeffd640 1 -- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a804de60 msgr2=0x7f26a8053410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26aeffd640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a804de60 0x7f26a8053410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26aeffd640 1 -- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 msgr2=0x7f26a80de510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26af7fe640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a80de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fdf56d9d640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf5806a4c0 0x7fdf581572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fdf56d9d640 1 -- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5806a950 msgr2=0x7fdf581642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fdf56d9d640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5806a950 0x7fdf581642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe610edd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 0x7fe60c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fdf56d9d640 1 -- 172.21.15.111:0/3727883122 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50009b30 con 0x7fdf58164820 2023-12-04T05:39:23.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fdf5659c640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf5806a4c0 0x7fdf581572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:23.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe610edd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 0x7fe60c154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58998/0 (socket says 172.21.15.111:58998) 2023-12-04T05:39:23.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe610edd640 1 -- 172.21.15.111:0/574530616 learned_addr learned my addr 172.21.15.111:0/574530616 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe60bfff640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60c160cc0 0x7fe60c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb6d9d640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb80b0580 0x7f0eb80b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60c1567f0 0x7fe60c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26aeffd640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a80de510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26aeffd640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80ddd10 con 0x7f26a817c1a0 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c2117a0 con 0x7fe60c15ac00 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c152b00 con 0x7fe60c1567f0 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c1552e0 con 0x7fe60c160cc0 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fe6116de640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60c1567f0 0x7fe60c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb659c640 1 -- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb816c830 msgr2=0x7f0eb816b480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb659c640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb816c830 0x7f0eb816b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb659c640 1 -- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb80b0580 msgr2=0x7f0eb80b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb659c640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb80b0580 0x7f0eb80b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb659c640 1 -- 172.21.15.111:0/1699163168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb81f3460 con 0x7f0ea00047c0 2023-12-04T05:39:23.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff88051d70 con 0x7eff880e3090 2023-12-04T05:39:23.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.606+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880d6810 con 0x7eff880e3090 2023-12-04T05:39:23.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000ca70 con 0x7eff880e3090 2023-12-04T05:39:23.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000ce00 con 0x7eff880e3090 2023-12-04T05:39:23.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f0eb659c640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 0x7f0eb81f2a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000cb60 tx=0x7f0eb000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7f26affff640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a804de60 0x7f26a8053410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:23.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fd8dad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4005e60 0x7fd8d4049c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fd8da59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 0x7fd8d40d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fd8da59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 0x7fd8d40d4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59014/0 (socket says 172.21.15.111:59014) 2023-12-04T05:39:23.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.607+0000 7fd8da59c640 1 -- 172.21.15.111:0/1944028598 learned_addr learned my addr 172.21.15.111:0/1944028598 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000ad40 con 0x7f0ea00047c0 2023-12-04T05:39:23.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f26aeffd640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 0x7f26a804d920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f269c010890 tx=0x7f269c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fd8db59e640 1 -- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4005e60 msgr2=0x7fd8d4049c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fd8db59e640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4005e60 0x7fd8d4049c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fd8db59e640 1 -- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 msgr2=0x7fd8d40d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fd8db59e640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 0x7fd8d40d4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fdf56d9d640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 0x7fdf5815db80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c3f0 tx=0x7fdf4800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480052d0 con 0x7fdf58164820 2023-12-04T05:39:23.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fd8db59e640 1 -- 172.21.15.111:0/1944028598 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc0094d0 con 0x7fd8d4053790 2023-12-04T05:39:23.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fd8da59c640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 0x7fd8d40d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fe6116de640 1 -- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60c160cc0 msgr2=0x7fe60c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fe6116de640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60c160cc0 0x7fe60c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fe6116de640 1 -- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 msgr2=0x7fe60c154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fe6116de640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 0x7fe60c154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fe6116de640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8009400 con 0x7fe60c1567f0 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fe610edd640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 0x7fe60c154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb0012400 con 0x7f0ea00047c0 2023-12-04T05:39:23.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000aea0 con 0x7f0ea00047c0 2023-12-04T05:39:23.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00cb40 con 0x7f26a817c1a0 2023-12-04T05:39:23.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00cca0 con 0x7f26a817c1a0 2023-12-04T05:39:23.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f26b5b05640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a80ddff0 con 0x7f26a817c1a0 2023-12-04T05:39:23.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff70005250 con 0x7eff880e3090 2023-12-04T05:39:23.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb81f3740 con 0x7f0ea00047c0 2023-12-04T05:39:23.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb81f3d00 con 0x7f0ea00047c0 2023-12-04T05:39:23.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c015690 con 0x7f26a817c1a0 2023-12-04T05:39:23.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.609+0000 7f26b5b05640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a817c620 con 0x7f26a817c1a0 2023-12-04T05:39:23.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.609+0000 7fd8db59e640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d4053790 0x7fd8d40e9220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc069da0 tx=0x7fd8dc067830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.608+0000 7fdf5c961640 1 -- 172.21.15.111:0/3727883122 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581ec1f0 con 0x7fdf58164820 2023-12-04T05:39:23.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.609+0000 7fdf5c961640 1 -- 172.21.15.111:0/3727883122 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581ec7b0 con 0x7fdf58164820 2023-12-04T05:39:23.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc06ba60 con 0x7fd8d4053790 2023-12-04T05:39:23.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48005450 con 0x7fdf58164820 2023-12-04T05:39:23.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48007260 con 0x7fdf58164820 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fe6116de640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60c1567f0 0x7fe60c15c340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe60400be20 tx=0x7fe60400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d40e2f00 con 0x7fd8d4053790 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d40e34c0 con 0x7fd8d4053790 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc06bc20 con 0x7fd8d4053790 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0721c0 con 0x7fd8d4053790 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60400ce70 con 0x7fe60c1567f0 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604014440 con 0x7fe60c1567f0 2023-12-04T05:39:23.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c15b160 con 0x7fe60c1567f0 2023-12-04T05:39:23.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c1558a0 con 0x7fe60c1567f0 2023-12-04T05:39:23.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604014810 con 0x7fe60c1567f0 2023-12-04T05:39:23.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0eb0007070 con 0x7f0ea00047c0 2023-12-04T05:39:23.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc005250 con 0x7fd8d4053790 2023-12-04T05:39:23.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7f0ea4ff9640 1 -- 172.21.15.111:0/1699163168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb80b02a0 con 0x7f0ea00047c0 2023-12-04T05:39:23.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.612+0000 7f0ea6ffd640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e9806b370 0x7f0e9806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.612+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0eb0089c60 con 0x7f0ea00047c0 2023-12-04T05:39:23.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.610+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2688005250 con 0x7f26a817c1a0 2023-12-04T05:39:23.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.611+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f269c0157f0 con 0x7f26a817c1a0 2023-12-04T05:39:23.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.612+0000 7f269bfff640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f269406b320 0x7f269406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.612+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fdf480073c0 con 0x7fdf58164820 2023-12-04T05:39:23.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.612+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe60401d3b0 con 0x7fe60c1567f0 2023-12-04T05:39:23.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.613+0000 7fdf46ffd640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdf3806b370 0x7fdf3806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.613+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf48088de0 con 0x7fdf58164820 2023-12-04T05:39:23.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.612+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7eff80013470 con 0x7eff880e3090 2023-12-04T05:39:23.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.613+0000 7eff85ffb640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff7406b3a0 0x7eff7406d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.613+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff80089d80 con 0x7eff880e3090 2023-12-04T05:39:23.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.613+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd8dc072320 con 0x7fd8d4053790 2023-12-04T05:39:23.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.613+0000 7fdf5c961640 1 -- 172.21.15.111:0/3727883122 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c005250 con 0x7fdf58164820 2023-12-04T05:39:23.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7fdf5659c640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdf3806b370 0x7fdf3806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7fd8c37fe640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd8c806b370 0x7fd8c806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8dc0b2e40 con 0x7fd8d4053790 2023-12-04T05:39:23.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7fe6097fa640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe5f006b370 0x7fe5f006d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7fe610edd640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe5f006b370 0x7fe5f006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7f0eb5d9b640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e9806b370 0x7f0e9806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7f26af7fe640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f269406b320 0x7f269406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.614+0000 7eff9555a640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff7406b3a0 0x7eff7406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7eff9555a640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff7406b3a0 0x7eff7406d860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff880062b0 tx=0x7eff9008d030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe604089cc0 con 0x7fe60c1567f0 2023-12-04T05:39:23.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7f0eb5d9b640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e9806b370 0x7f0e9806d830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8036000 tx=0x7f0ea8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7f26af7fe640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f269406b320 0x7f269406d7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26a4007fd0 tx=0x7f26a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7fdf5659c640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdf3806b370 0x7fdf3806d830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf50035000 tx=0x7fdf50030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c2118e0 con 0x7fe60c1567f0 2023-12-04T05:39:23.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.615+0000 7fd8dad9d640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd8c806b370 0x7fd8c806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.616+0000 7fe610edd640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe5f006b370 0x7fe5f006d830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe600000f60 tx=0x7fe600009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.616+0000 7fd8dad9d640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd8c806b370 0x7fd8c806d830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0003610 tx=0x7fd8d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.616+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f269c010e80 con 0x7f26a817c1a0 2023-12-04T05:39:23.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.616+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff80054bd0 con 0x7eff880e3090 2023-12-04T05:39:23.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.617+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8dc0ce690 con 0x7fd8d4053790 2023-12-04T05:39:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.617+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0eb80b02a0 con 0x7f0ea00047c0 2023-12-04T05:39:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.622+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe60c2118e0 con 0x7fe60c1567f0 2023-12-04T05:39:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.622+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf48053d00 con 0x7fdf58164820 2023-12-04T05:39:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.624+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f269c055390 con 0x7f26a817c1a0 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.638+0000 7f982a651640 1 Processor -- start 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.638+0000 7f982a651640 1 -- start start 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f982a651640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f982a651640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f98240cba10 0x7f982406ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f982a651640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98240cd150 con 0x7f98240ccbf0 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f982a651640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98240cd430 con 0x7f98240cba10 2023-12-04T05:39:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f982a651640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98240cd710 con 0x7f982415af30 2023-12-04T05:39:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f6649659640 1 Processor -- start 2023-12-04T05:39:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f6649659640 1 -- start start 2023-12-04T05:39:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f98237fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f98240ccbf0 0x7f982415eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43152/0 (socket says 172.21.15.111:43152) 2023-12-04T05:39:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f98237fe640 1 -- 172.21.15.111:0/315572498 learned_addr learned my addr 172.21.15.111:0/315572498 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.640+0000 7f9823fff640 1 --2- 172.21.15.111:0/315572498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9828bee640 1 --2- 172.21.15.111:0/315572498 >> v2:172.21.15.192:3300/0 conn(0x7f98240cba10 0x7f982406ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9822ffd640 1 -- 172.21.15.111:0/315572498 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897001036 0 0) 0x7f98240cd150 con 0x7f98240ccbf0 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9822ffd640 1 -- 172.21.15.111:0/315572498 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9810003520 con 0x7f98240ccbf0 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f6649659640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c0e3e40 0x7f663c0e4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f6649659640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f663c03a7c0 0x7f663c0e4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f6649659640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f663c0e4d70 0x7f663c0072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f6649659640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c050380 con 0x7f663c0e4d70 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f6649659640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0062a0 con 0x7f663c03a7c0 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f6649659640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0064f0 con 0x7f663c0e3e40 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f6643fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c0e3e40 0x7f663c0e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f6643fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c0e3e40 0x7f663c0e4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:40310/0 (socket says 172.21.15.111:40310) 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f6643fff640 1 -- 172.21.15.111:0/4225067364 learned_addr learned my addr 172.21.15.111:0/4225067364 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9823fff640 1 -- 172.21.15.111:0/315572498 >> v2:172.21.15.192:3300/0 conn(0x7f98240cba10 msgr2=0x7f982406ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9823fff640 1 --2- 172.21.15.111:0/315572498 >> v2:172.21.15.192:3300/0 conn(0x7f98240cba10 0x7f982406ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f6648e58640 1 --2- 172.21.15.111:0/4225067364 >> v2:172.21.15.111:3300/0 conn(0x7f663c0e4d70 0x7f663c0072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9823fff640 1 -- 172.21.15.111:0/315572498 >> v1:172.21.15.192:6789/0 conn(0x7f98240ccbf0 legacy=0x7f982415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.641+0000 7f9823fff640 1 -- 172.21.15.111:0/315572498 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240cd9f0 con 0x7f982415af30 2023-12-04T05:39:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f9823fff640 1 --2- 172.21.15.111:0/315572498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982406b4d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f980c0097a0 tx=0x7f980c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2a27f56ca4982b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f6648e58640 1 -- 172.21.15.111:0/4225067364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c0e3e40 msgr2=0x7f663c0e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f6648e58640 1 --2- 172.21.15.111:0/4225067364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c0e3e40 0x7f663c0e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f6648e58640 1 -- 172.21.15.111:0/4225067364 >> v2:172.21.15.192:3300/0 conn(0x7f663c03a7c0 msgr2=0x7f663c0e4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:23.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f6648e58640 1 --2- 172.21.15.111:0/4225067364 >> v2:172.21.15.192:3300/0 conn(0x7f663c03a7c0 0x7f663c0e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f6648e58640 1 -- 172.21.15.111:0/4225067364 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0067d0 con 0x7f663c0e4d70 2023-12-04T05:39:23.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f9822ffd640 1 -- 172.21.15.111:0/315572498 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c02e070 con 0x7f982415af30 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f9822ffd640 1 -- 172.21.15.111:0/315572498 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c0315a0 con 0x7f982415af30 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.642+0000 7f9822ffd640 1 -- 172.21.15.111:0/315572498 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c031910 con 0x7f982415af30 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f982a651640 1 -- 172.21.15.111:0/315572498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 msgr2=0x7f982406b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.643+0000 7f982a651640 1 --2- 172.21.15.111:0/315572498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982406b4d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f980c0097a0 tx=0x7f980c02b840 comp rx=0 tx=0).stop 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f6648e58640 1 --2- 172.21.15.111:0/4225067364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c0e4d70 0x7f663c0072b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f663c008950 tx=0x7f663802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758c8ba8e6199aac server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f6642ffd640 1 -- 172.21.15.111:0/4225067364 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638031750 con 0x7f663c0e4d70 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f6642ffd640 1 -- 172.21.15.111:0/4225067364 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66380318b0 con 0x7f663c0e4d70 2023-12-04T05:39:23.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f6642ffd640 1 -- 172.21.15.111:0/4225067364 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638031c20 con 0x7f663c0e4d70 2023-12-04T05:39:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f982a651640 1 -- 172.21.15.111:0/315572498 shutdown_connections 2023-12-04T05:39:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f982a651640 1 --2- 172.21.15.111:0/315572498 >> v2:172.21.15.192:3300/0 conn(0x7f98240cba10 0x7f982406ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f982a651640 1 --2- 172.21.15.111:0/315572498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982406b4d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.644+0000 7f982a651640 1 -- 172.21.15.111:0/315572498 >> 172.21.15.111:0/315572498 conn(0x7f9824070cd0 msgr2=0x7f98240710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f982a651640 1 -- 172.21.15.111:0/315572498 shutdown_connections 2023-12-04T05:39:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 -- 172.21.15.111:0/4225067364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c0e4d70 msgr2=0x7f663c0072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 --2- 172.21.15.111:0/4225067364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c0e4d70 0x7f663c0072b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f663c008950 tx=0x7f663802b7f0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 -- 172.21.15.111:0/4225067364 shutdown_connections 2023-12-04T05:39:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 --2- 172.21.15.111:0/4225067364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c0e4d70 0x7f663c0072b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 --2- 172.21.15.111:0/4225067364 >> v2:172.21.15.192:3300/0 conn(0x7f663c03a7c0 0x7f663c0e4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 --2- 172.21.15.111:0/4225067364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c0e3e40 0x7f663c0e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f6649659640 1 -- 172.21.15.111:0/4225067364 >> 172.21.15.111:0/4225067364 conn(0x7f663c054ed0 msgr2=0x7f663c0552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.646+0000 7f6649659640 1 -- 172.21.15.111:0/4225067364 shutdown_connections 2023-12-04T05:39:23.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.645+0000 7f982a651640 1 -- 172.21.15.111:0/315572498 wait complete. 2023-12-04T05:39:23.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.646+0000 7f6649659640 1 -- 172.21.15.111:0/4225067364 wait complete. 2023-12-04T05:39:23.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f6649659640 1 Processor -- start 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.646+0000 7f982a651640 1 Processor -- start 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.647+0000 7f982a651640 1 -- start start 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f982a651640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240cba10 0x7f982415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f982a651640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98240ccbf0 0x7f982415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f982a651640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982412f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f982a651640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98240b2370 con 0x7f98240ccbf0 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f982a651640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9824140ff0 con 0x7f98240cba10 2023-12-04T05:39:23.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f982a651640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98241415b0 con 0x7f982415af30 2023-12-04T05:39:23.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f6649659640 1 -- start start 2023-12-04T05:39:23.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6649659640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c03a7c0 0x7f663c186ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6649659640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0e4d70 0x7f663c0495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f9823fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240cba10 0x7f982415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f9823fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240cba10 0x7f982415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57332/0 (socket says 172.21.15.111:57332) 2023-12-04T05:39:23.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6649659640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 0x7f663c0e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6649659640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c187370 con 0x7f663c04f3a0 2023-12-04T05:39:23.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6649659640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c187650 con 0x7f663c0e4d70 2023-12-04T05:39:23.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6649659640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c187930 con 0x7f663c03a7c0 2023-12-04T05:39:23.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f9823fff640 1 -- 172.21.15.111:0/2085654888 learned_addr learned my addr 172.21.15.111:0/2085654888 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6648e58640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 0x7f663c0e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6648e58640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 0x7f663c0e20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59028/0 (socket says 172.21.15.111:59028) 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6648e58640 1 -- 172.21.15.111:0/4025071965 learned_addr learned my addr 172.21.15.111:0/4025071965 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f6643fff640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c03a7c0 0x7f663c186ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f6648e58640 1 -- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c03a7c0 msgr2=0x7f663c186ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f6648e58640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c03a7c0 0x7f663c186ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f6648e58640 1 -- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0e4d70 msgr2=0x7f663c0495d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f6648e58640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0e4d70 0x7f663c0495d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f6648e58640 1 -- 172.21.15.111:0/4025071965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638009400 con 0x7f663c04f3a0 2023-12-04T05:39:23.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f6648e58640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 0x7f663c0e20e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6638009890 tx=0x7f66380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.648+0000 7f9828bee640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982412f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f9828bee640 1 -- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240cba10 msgr2=0x7f982415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f9828bee640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240cba10 0x7f982415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f9828bee640 1 -- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98240ccbf0 msgr2=0x7f982415d8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f98237fe640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98240ccbf0 0x7f982415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f9828bee640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98240ccbf0 0x7f982415d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f9828bee640 1 -- 172.21.15.111:0/2085654888 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c009400 con 0x7f982415af30 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.649+0000 7f9828bee640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982412f870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f981c00eba0 tx=0x7f981c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f98237fe640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98240ccbf0 0x7f982415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:23.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c00cd20 con 0x7f982415af30 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.650+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c00f040 con 0x7f982415af30 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f982a651640 1 -- 172.21.15.111:0/2085654888 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982415bc10 con 0x7f982415af30 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.651+0000 7f982a651640 1 -- 172.21.15.111:0/2085654888 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9824141240 con 0x7f982415af30 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.652+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0146c0 con 0x7f982415af30 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.653+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638006dd0 con 0x7f663c04f3a0 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.653+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6638003610 con 0x7f663c04f3a0 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.653+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66380039c0 con 0x7f663c04f3a0 2023-12-04T05:39:23.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.654+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c03adc0 con 0x7f663c04f3a0 2023-12-04T05:39:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.654+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c187ef0 con 0x7f663c04f3a0 2023-12-04T05:39:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.654+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f981c014820 con 0x7f982415af30 2023-12-04T05:39:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.654+0000 7f9820ff9640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f980406b2f0 0x7f980406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.654+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f981c08a6f0 con 0x7f982415af30 2023-12-04T05:39:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.655+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98240cc0c0 con 0x7f982415af30 2023-12-04T05:39:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.656+0000 7f9823fff640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f980406b2f0 0x7f980406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.656+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f663803a070 con 0x7f663c04f3a0 2023-12-04T05:39:23.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.657+0000 7f9823fff640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f980406b2f0 0x7f980406d7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f980c000c00 tx=0x7f980c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.657+0000 7f6640ff9640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f662806b370 0x7f662806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:23.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.657+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c188110 con 0x7f663c04f3a0 2023-12-04T05:39:23.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.663+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f981c018080 con 0x7f982415af30 2023-12-04T05:39:23.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.666+0000 7f6643fff640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f662806b370 0x7f662806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:23.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.667+0000 7f6643fff640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f662806b370 0x7f662806d830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f664402cf20 tx=0x7f664408af90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:23.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.667+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66380ab8e0 con 0x7f663c04f3a0 2023-12-04T05:39:23.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:23.668+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66380af240 con 0x7f663c04f3a0 2023-12-04T05:39:24.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.014+0000 7fdf5c961640 1 -- 172.21.15.111:0/3727883122 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdf3c005500 con 0x7fdf58164820 2023-12-04T05:39:24.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.015+0000 7fdf46ffd640 1 -- 172.21.15.111:0/3727883122 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf48057320 con 0x7fdf58164820 2023-12-04T05:39:24.016 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2023-12-04T05:39:24.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.017+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff70005500 con 0x7eff880e3090 2023-12-04T05:39:24.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7eff85ffb640 1 -- 172.21.15.111:0/4147956536 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff80017090 con 0x7eff880e3090 2023-12-04T05:39:24.019 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 -- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdf3806b370 msgr2=0x7fdf3806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdf3806b370 0x7fdf3806d830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf50035000 tx=0x7fdf50030000 comp rx=0 tx=0).stop 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 -- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 msgr2=0x7fdf5815db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 0x7fdf5815db80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c3f0 tx=0x7fdf4800c8b0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 -- 172.21.15.111:0/3727883122 shutdown_connections 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdf3806b370 0x7fdf3806d830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf58164820 0x7fdf5815db80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5806a950 0x7fdf581642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 --2- 172.21.15.111:0/3727883122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf5806a4c0 0x7fdf581572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.018+0000 7fdf44ff9640 1 -- 172.21.15.111:0/3727883122 >> 172.21.15.111:0/3727883122 conn(0x7fdf580b1310 msgr2=0x7fdf580b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.019+0000 7fdf44ff9640 1 -- 172.21.15.111:0/3727883122 shutdown_connections 2023-12-04T05:39:24.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.019+0000 7fdf44ff9640 1 -- 172.21.15.111:0/3727883122 wait complete. 2023-12-04T05:39:24.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.019+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2688005500 con 0x7f26a817c1a0 2023-12-04T05:39:24.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.020+0000 7f269bfff640 1 -- 172.21.15.111:0/768774721 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f269c00e050 con 0x7f26a817c1a0 2023-12-04T05:39:24.022 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2023-12-04T05:39:24.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff7406b3a0 msgr2=0x7eff7406d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff7406b3a0 0x7eff7406d860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff880062b0 tx=0x7eff9008d030 comp rx=0 tx=0).stop 2023-12-04T05:39:24.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe60c2118e0 con 0x7fe60c1567f0 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 msgr2=0x7eff8803dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 0x7eff8803dcf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff8000eab0 tx=0x7eff8000ef70 comp rx=0 tx=0).stop 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 shutdown_connections 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff7406b3a0 0x7eff7406d860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff880e3090 0x7eff8803dcf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff880542d0 0x7eff8803d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 --2- 172.21.15.111:0/4147956536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff8803e810 0x7eff880d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.022+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 >> 172.21.15.111:0/4147956536 conn(0x7eff880559b0 msgr2=0x7eff88039d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 shutdown_connections 2023-12-04T05:39:24.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7eff9655c640 1 -- 172.21.15.111:0/4147956536 wait complete. 2023-12-04T05:39:24.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7fe6097fa640 1 -- 172.21.15.111:0/574530616 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe60c2118e0 con 0x7fe60c1567f0 2023-12-04T05:39:24.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f269406b320 msgr2=0x7f269406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f2699ffb640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f269406b320 0x7f269406d7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26a4007fd0 tx=0x7f26a4009040 comp rx=0 tx=0).stop 2023-12-04T05:39:24.026 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:24.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 msgr2=0x7f26a804d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f2699ffb640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 0x7f26a804d920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f269c010890 tx=0x7f269c010d50 comp rx=0 tx=0).stop 2023-12-04T05:39:24.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f0ea4ff9640 1 -- 172.21.15.111:0/1699163168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0eb80b02a0 con 0x7f0ea00047c0 2023-12-04T05:39:24.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd8bc0057d0 con 0x7fd8d4053790 2023-12-04T05:39:24.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 shutdown_connections 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.023+0000 7f2699ffb640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f269406b320 0x7f269406d7e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f2699ffb640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26a804de60 0x7f26a8053410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f2699ffb640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26a817c1a0 0x7f26a804d920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f2699ffb640 1 --2- 172.21.15.111:0/768774721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26a0004d00 0x7f26a80de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 >> 172.21.15.111:0/768774721 conn(0x7f26a8009f70 msgr2=0x7f26a8047260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f0ea6ffd640 1 -- 172.21.15.111:0/1699163168 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0eb80b02a0 con 0x7f0ea00047c0 2023-12-04T05:39:24.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7fd8c37fe640 1 -- 172.21.15.111:0/1944028598 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd8dc101e60 con 0x7fd8d4053790 2023-12-04T05:39:24.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 shutdown_connections 2023-12-04T05:39:24.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.024+0000 7f2699ffb640 1 -- 172.21.15.111:0/768774721 wait complete. 2023-12-04T05:39:24.030 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:24.030 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:24.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe5f006b370 msgr2=0x7fe5f006d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe5f006b370 0x7fe5f006d830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe600000f60 tx=0x7fe600009040 comp rx=0 tx=0).stop 2023-12-04T05:39:24.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60c1567f0 msgr2=0x7fe60c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60c1567f0 0x7fe60c15c340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe60400be20 tx=0x7fe60400bf10 comp rx=0 tx=0).stop 2023-12-04T05:39:24.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 shutdown_connections 2023-12-04T05:39:24.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e9806b370 msgr2=0x7f0e9806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e9806b370 0x7f0e9806d830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8036000 tx=0x7f0ea8009030 comp rx=0 tx=0).stop 2023-12-04T05:39:24.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe5f006b370 0x7fe5f006d830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 msgr2=0x7f0eb81f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd8c806b370 msgr2=0x7fd8c806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 0x7f0eb81f2a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000cb60 tx=0x7f0eb000a430 comp rx=0 tx=0).stop 2023-12-04T05:39:24.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fd8e086a640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd8c806b370 0x7fd8c806d830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0003610 tx=0x7fd8d0008040 comp rx=0 tx=0).stop 2023-12-04T05:39:24.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60c1567f0 0x7fe60c15c340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60c160cc0 0x7fe60c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d4053790 msgr2=0x7fd8d40e9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.027+0000 7fd8e086a640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d4053790 0x7fd8d40e9220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc069da0 tx=0x7fd8dc067830 comp rx=0 tx=0).stop 2023-12-04T05:39:24.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 shutdown_connections 2023-12-04T05:39:24.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e9806b370 0x7f0e9806d830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb80b0580 0x7f0eb80b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 shutdown_connections 2023-12-04T05:39:24.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb816c830 0x7f0eb816b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fd8e086a640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd8c806b370 0x7fd8c806d830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 --2- 172.21.15.111:0/1699163168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00047c0 0x7f0eb81f2a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fd8e086a640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8d4053790 0x7fd8d40e9220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 >> 172.21.15.111:0/1699163168 conn(0x7f0eb80d0dd0 msgr2=0x7f0eb815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fd8e086a640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8d404f8d0 0x7fd8d40d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fd8e086a640 1 --2- 172.21.15.111:0/1944028598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8d4005e60 0x7fd8d4049c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 >> 172.21.15.111:0/1944028598 conn(0x7fd8d4009f80 msgr2=0x7fd8d40e82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fe611edf640 1 --2- 172.21.15.111:0/574530616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c15ac00 0x7fe60c154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 >> 172.21.15.111:0/574530616 conn(0x7fe60c070a80 msgr2=0x7fe60c06d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 shutdown_connections 2023-12-04T05:39:24.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7fe611edf640 1 -- 172.21.15.111:0/574530616 wait complete. 2023-12-04T05:39:24.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 shutdown_connections 2023-12-04T05:39:24.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.028+0000 7f0eb759e640 1 -- 172.21.15.111:0/1699163168 wait complete. 2023-12-04T05:39:24.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.029+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 shutdown_connections 2023-12-04T05:39:24.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.029+0000 7fd8e086a640 1 -- 172.21.15.111:0/1944028598 wait complete. 2023-12-04T05:39:24.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-12-04T05:39:24.044 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-12-04T05:39:24.045 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-12-04T05:39:24.047 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-12-04T05:39:24.051 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-12-04T05:39:24.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.050+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9824141740 con 0x7f982415af30 2023-12-04T05:39:24.052 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-12-04T05:39:24.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.050+0000 7f9820ff9640 1 -- 172.21.15.111:0/2085654888 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f981c055640 con 0x7f982415af30 2023-12-04T05:39:24.053 INFO:teuthology.orchestra.run.smithi111.stdout:55834574855 2023-12-04T05:39:24.054 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f980406b2f0 msgr2=0x7f980406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f980406b2f0 0x7f980406d7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f980c000c00 tx=0x7f980c008030 comp rx=0 tx=0).stop 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 msgr2=0x7f982412f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982412f870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f981c00eba0 tx=0x7f981c00c490 comp rx=0 tx=0).stop 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 shutdown_connections 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f980406b2f0 0x7f980406d7b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415af30 0x7f982412f870 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98240ccbf0 0x7f982415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 --2- 172.21.15.111:0/2085654888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240cba10 0x7f982415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 >> 172.21.15.111:0/2085654888 conn(0x7f9824070cd0 msgr2=0x7f98241512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 shutdown_connections 2023-12-04T05:39:24.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.052+0000 7f981a7fc640 1 -- 172.21.15.111:0/2085654888 wait complete. 2023-12-04T05:39:24.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.053+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f663c008de0 con 0x7f663c04f3a0 2023-12-04T05:39:24.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.054+0000 7f6640ff9640 1 -- 172.21.15.111:0/4025071965 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6638038090 con 0x7f663c04f3a0 2023-12-04T05:39:24.057 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f662806b370 msgr2=0x7f662806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f662806b370 0x7f662806d830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f664402cf20 tx=0x7f664408af90 comp rx=0 tx=0).stop 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 msgr2=0x7f663c0e20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 0x7f663c0e20e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6638009890 tx=0x7f66380046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 shutdown_connections 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f662806b370 0x7f662806d830 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663c04f3a0 0x7f663c0e20e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0e4d70 0x7f663c0495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 --2- 172.21.15.111:0/4025071965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f663c03a7c0 0x7f663c186ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 >> 172.21.15.111:0/4025071965 conn(0x7f663c054ed0 msgr2=0x7f663c053cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 shutdown_connections 2023-12-04T05:39:24.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.056+0000 7f6649659640 1 -- 172.21.15.111:0/4025071965 wait complete. 2023-12-04T05:39:24.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-12-04T05:39:24.065 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-12-04T05:39:24.068 DEBUG:teuthology.parallel:result is None 2023-12-04T05:39:24.068 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-04T05:39:24.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:39:24.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.206+0000 7f8119851640 1 Processor -- start 2023-12-04T05:39:24.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8119851640 1 -- start start 2023-12-04T05:39:24.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8119851640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f81140b5b30 0x7f811406ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8119851640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8114144290 con 0x7f8114143eb0 2023-12-04T05:39:24.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8119851640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8114144cd0 con 0x7f81140ce420 2023-12-04T05:39:24.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8119851640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81140cff20 con 0x7f81140b5b30 2023-12-04T05:39:24.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8112ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8114143eb0 0x7f81140b5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58776/0 (socket says 172.21.15.111:58776) 2023-12-04T05:39:24.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.207+0000 7f8112ffd640 1 -- 172.21.15.111:0/67339063 learned_addr learned my addr 172.21.15.111:0/67339063 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f81127fc640 1 --2- 172.21.15.111:0/67339063 >> v2:172.21.15.192:3300/0 conn(0x7f81140b5b30 0x7f811406ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127702935 0 0) 0x7f8104001490 con 0x7f8114143eb0 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80fc007930 con 0x7f8114143eb0 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082235666 0 0) 0x7f80fc007930 con 0x7f8114143eb0 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 >> v2:172.21.15.192:3300/0 conn(0x7f81140b5b30 msgr2=0x7f811406ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 --2- 172.21.15.111:0/67339063 >> v2:172.21.15.192:3300/0 conn(0x7f81140b5b30 0x7f811406ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 >> v1:172.21.15.192:6789/0 conn(0x7f81140ce420 legacy=0x7f811415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114144580 con 0x7f8114143eb0 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f8104001b80 con 0x7f8114143eb0 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 >> v1:172.21.15.111:6789/0 conn(0x7f8114143eb0 legacy=0x7f81140b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc007d80 0x7f80fc0081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80fc008bd0 0x7f80fc009050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc009590 0x7f80fc00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81140cff20 con 0x7f80fc007d80 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc010530 con 0x7f80fc008bd0 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0106f0 con 0x7f80fc009590 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8112ffd640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc007d80 0x7f80fc0081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f81127fc640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80fc008bd0 0x7f80fc009050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f81137fe640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc009590 0x7f80fc00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8112ffd640 1 -- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc009590 msgr2=0x7f80fc00fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8112ffd640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc009590 0x7f80fc00fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8112ffd640 1 -- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80fc008bd0 msgr2=0x7f80fc009050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8112ffd640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80fc008bd0 0x7f80fc009050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f81137fe640 1 -- 172.21.15.111:0/67339063 reap_dead start 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.208+0000 7f8112ffd640 1 -- 172.21.15.111:0/67339063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc007930 con 0x7f80fc007d80 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.209+0000 7f8112ffd640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc007d80 0x7f80fc0081a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f810400a830 tx=0x7f80fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c647f3e0236611 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.209+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104034630 con 0x7f80fc007d80 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.209+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810402f070 con 0x7f80fc007d80 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.209+0000 7f8111ffb640 1 -- 172.21.15.111:0/67339063 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104034a30 con 0x7f80fc007d80 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 -- 172.21.15.111:0/67339063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc007d80 msgr2=0x7f80fc0081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc007d80 0x7f80fc0081a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f810400a830 tx=0x7f80fc009c00 comp rx=0 tx=0).stop 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 -- 172.21.15.111:0/67339063 shutdown_connections 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 --2- 172.21.15.111:0/67339063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc007d80 0x7f80fc0081a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 -- 172.21.15.111:0/67339063 >> 172.21.15.111:0/67339063 conn(0x7f81140b1e10 msgr2=0x7f81140b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 -- 172.21.15.111:0/67339063 shutdown_connections 2023-12-04T05:39:24.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.210+0000 7f8119851640 1 -- 172.21.15.111:0/67339063 wait complete. 2023-12-04T05:39:24.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.211+0000 7f8119851640 1 Processor -- start 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.211+0000 7f8119851640 1 -- start start 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8119851640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc007d80 0x7f8114142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8119851640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8114156dd0 0x7f811415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8119851640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 0x7f8114071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8119851640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811415d330 con 0x7f811415bee0 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8119851640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811415d600 con 0x7f8114156dd0 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8119851640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811415d8e0 con 0x7f80fc007d80 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 0x7f8114071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 0x7f8114071390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59060/0 (socket says 172.21.15.111:59060) 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 -- 172.21.15.111:0/789910199 learned_addr learned my addr 172.21.15.111:0/789910199 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f8112ffd640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc007d80 0x7f8114142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81127fc640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8114156dd0 0x7f811415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 -- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc007d80 msgr2=0x7f8114142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc007d80 0x7f8114142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 -- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8114156dd0 msgr2=0x7f811415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8114156dd0 0x7f811415cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 -- 172.21.15.111:0/789910199 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811415dbc0 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.212+0000 7f81137fe640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 0x7f8114071390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f81140cf6c0 tx=0x7f810c00f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.213+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c0052e0 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.213+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c005440 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.213+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c003830 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.213+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811415b6e0 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.213+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8114157250 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.214+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811415bc00 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.214+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f810c003990 con 0x7f811415bee0 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.214+0000 7f80fbfff640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f810006b2d0 0x7f810006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.214+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f810c089510 con 0x7f811415bee0 2023-12-04T05:39:24.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.214+0000 7f8112ffd640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f810006b2d0 0x7f810006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.215+0000 7f8112ffd640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f810006b2d0 0x7f810006d790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8104034000 tx=0x7f810402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.219+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f811415bc00 con 0x7f811415bee0 2023-12-04T05:39:24.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.383+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f811415da50 con 0x7f810006b2d0 2023-12-04T05:39:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.389+0000 7f80fbfff640 1 -- 172.21.15.111:0/789910199 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264464 (secure 0 0 0) 0x7f811415da50 con 0x7f810006b2d0 2023-12-04T05:39:24.393 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:24.395 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:39:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f810006b2d0 msgr2=0x7f810006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f810006b2d0 0x7f810006d790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8104034000 tx=0x7f810402f000 comp rx=0 tx=0).stop 2023-12-04T05:39:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 msgr2=0x7f8114071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 0x7f8114071390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f81140cf6c0 tx=0x7f810c00f840 comp rx=0 tx=0).stop 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 shutdown_connections 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f810006b2d0 0x7f810006d790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f811415bee0 0x7f8114071390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8114156dd0 0x7f811415cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 --2- 172.21.15.111:0/789910199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80fc007d80 0x7f8114142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.392+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 >> 172.21.15.111:0/789910199 conn(0x7f81140b1e10 msgr2=0x7f81140b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.393+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 shutdown_connections 2023-12-04T05:39:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.393+0000 7f8119851640 1 -- 172.21.15.111:0/789910199 wait complete. 2023-12-04T05:39:24.405 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-04T05:39:22.473710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.272803"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519836+0000","last_change":"2023-12-04T05:39:17.207470+0000","last_active":"2023-12-04T05:39:18.519836+0000","last_peered":"2023-12-04T05:39:18.519836+0000","last_clean":"2023-12-04T05:39:18.519836+0000","last_became_active":"2023-12-04T05:39:17.207195+0000","last_became_peered":"2023-12-04T05:39:17.207195+0000","last_unstale":"2023-12-04T05:39:18.519836+0000","last_undegraded":"2023-12-04T05:39:18.519836+0000","last_fullsized":"2023-12-04T05:39:18.519836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:01:28.140633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195840+0000","last_change":"2023-12-04T05:39:17.210099+0000","last_active":"2023-12-04T05:39:18.195840+0000","last_peered":"2023-12-04T05:39:18.195840+0000","last_clean":"2023-12-04T05:39:18.195840+0000","last_became_active":"2023-12-04T05:39:17.209800+0000","last_became_peered":"2023-12-04T05:39:17.209800+0000","last_unstale":"2023-12-04T05:39:18.195840+0000","last_undegraded":"2023-12-04T05:39:18.195840+0000","last_fullsized":"2023-12-04T05:39:18.195840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518434+0000","last_change":"2023-12-04T05:39:17.213084+0000","last_active":"2023-12-04T05:39:18.518434+0000","last_peered":"2023-12-04T05:39:18.518434+0000","last_clean":"2023-12-04T05:39:18.518434+0000","last_became_active":"2023-12-04T05:39:17.212742+0000","last_became_peered":"2023-12-04T05:39:17.212742+0000","last_unstale":"2023-12-04T05:39:18.518434+0000","last_undegraded":"2023-12-04T05:39:18.518434+0000","last_fullsized":"2023-12-04T05:39:18.518434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195184+0000","last_change":"2023-12-04T05:39:17.209715+0000","last_active":"2023-12-04T05:39:18.195184+0000","last_peered":"2023-12-04T05:39:18.195184+0000","last_clean":"2023-12-04T05:39:18.195184+0000","last_became_active":"2023-12-04T05:39:17.209399+0000","last_became_peered":"2023-12-04T05:39:17.209399+0000","last_unstale":"2023-12-04T05:39:18.195184+0000","last_undegraded":"2023-12-04T05:39:18.195184+0000","last_fullsized":"2023-12-04T05:39:18.195184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519183+0000","last_change":"2023-12-04T05:39:17.209067+0000","last_active":"2023-12-04T05:39:18.519183+0000","last_peered":"2023-12-04T05:39:18.519183+0000","last_clean":"2023-12-04T05:39:18.519183+0000","last_became_active":"2023-12-04T05:39:17.208794+0000","last_became_peered":"2023-12-04T05:39:17.208794+0000","last_unstale":"2023-12-04T05:39:18.519183+0000","last_undegraded":"2023-12-04T05:39:18.519183+0000","last_fullsized":"2023-12-04T05:39:18.519183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195035+0000","last_change":"2023-12-04T05:39:17.216846+0000","last_active":"2023-12-04T05:39:18.195035+0000","last_peered":"2023-12-04T05:39:18.195035+0000","last_clean":"2023-12-04T05:39:18.195035+0000","last_became_active":"2023-12-04T05:39:17.216536+0000","last_became_peered":"2023-12-04T05:39:17.216536+0000","last_unstale":"2023-12-04T05:39:18.195035+0000","last_undegraded":"2023-12-04T05:39:18.195035+0000","last_fullsized":"2023-12-04T05:39:18.195035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:53.681504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519109+0000","last_change":"2023-12-04T05:39:17.216228+0000","last_active":"2023-12-04T05:39:18.519109+0000","last_peered":"2023-12-04T05:39:18.519109+0000","last_clean":"2023-12-04T05:39:18.519109+0000","last_became_active":"2023-12-04T05:39:17.215958+0000","last_became_peered":"2023-12-04T05:39:17.215958+0000","last_unstale":"2023-12-04T05:39:18.519109+0000","last_undegraded":"2023-12-04T05:39:18.519109+0000","last_fullsized":"2023-12-04T05:39:18.519109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495336+0000","last_change":"2023-12-04T05:39:17.212872+0000","last_active":"2023-12-04T05:39:18.495336+0000","last_peered":"2023-12-04T05:39:18.495336+0000","last_clean":"2023-12-04T05:39:18.495336+0000","last_became_active":"2023-12-04T05:39:17.212550+0000","last_became_peered":"2023-12-04T05:39:17.212550+0000","last_unstale":"2023-12-04T05:39:18.495336+0000","last_undegraded":"2023-12-04T05:39:18.495336+0000","last_fullsized":"2023-12-04T05:39:18.495336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519658+0000","last_change":"2023-12-04T05:39:17.210290+0000","last_active":"2023-12-04T05:39:18.519658+0000","last_peered":"2023-12-04T05:39:18.519658+0000","last_clean":"2023-12-04T05:39:18.519658+0000","last_became_active":"2023-12-04T05:39:17.210041+0000","last_became_peered":"2023-12-04T05:39:17.210041+0000","last_unstale":"2023-12-04T05:39:18.519658+0000","last_undegraded":"2023-12-04T05:39:18.519658+0000","last_fullsized":"2023-12-04T05:39:18.519658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:01:33.532203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519694+0000","last_change":"2023-12-04T05:39:17.212608+0000","last_active":"2023-12-04T05:39:18.519694+0000","last_peered":"2023-12-04T05:39:18.519694+0000","last_clean":"2023-12-04T05:39:18.519694+0000","last_became_active":"2023-12-04T05:39:17.212249+0000","last_became_peered":"2023-12-04T05:39:17.212249+0000","last_unstale":"2023-12-04T05:39:18.519694+0000","last_undegraded":"2023-12-04T05:39:18.519694+0000","last_fullsized":"2023-12-04T05:39:18.519694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519356+0000","last_change":"2023-12-04T05:39:17.206680+0000","last_active":"2023-12-04T05:39:18.519356+0000","last_peered":"2023-12-04T05:39:18.519356+0000","last_clean":"2023-12-04T05:39:18.519356+0000","last_became_active":"2023-12-04T05:39:17.206326+0000","last_became_peered":"2023-12-04T05:39:17.206326+0000","last_unstale":"2023-12-04T05:39:18.519356+0000","last_undegraded":"2023-12-04T05:39:18.519356+0000","last_fullsized":"2023-12-04T05:39:18.519356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194773+0000","last_change":"2023-12-04T05:39:17.206360+0000","last_active":"2023-12-04T05:39:18.194773+0000","last_peered":"2023-12-04T05:39:18.194773+0000","last_clean":"2023-12-04T05:39:18.194773+0000","last_became_active":"2023-12-04T05:39:17.205989+0000","last_became_peered":"2023-12-04T05:39:17.205989+0000","last_unstale":"2023-12-04T05:39:18.194773+0000","last_undegraded":"2023-12-04T05:39:18.194773+0000","last_fullsized":"2023-12-04T05:39:18.194773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:22:45.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495552+0000","last_change":"2023-12-04T05:39:17.207638+0000","last_active":"2023-12-04T05:39:18.495552+0000","last_peered":"2023-12-04T05:39:18.495552+0000","last_clean":"2023-12-04T05:39:18.495552+0000","last_became_active":"2023-12-04T05:39:17.207368+0000","last_became_peered":"2023-12-04T05:39:17.207368+0000","last_unstale":"2023-12-04T05:39:18.495552+0000","last_undegraded":"2023-12-04T05:39:18.495552+0000","last_fullsized":"2023-12-04T05:39:18.495552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518614+0000","last_change":"2023-12-04T05:39:17.210284+0000","last_active":"2023-12-04T05:39:18.518614+0000","last_peered":"2023-12-04T05:39:18.518614+0000","last_clean":"2023-12-04T05:39:18.518614+0000","last_became_active":"2023-12-04T05:39:17.209989+0000","last_became_peered":"2023-12-04T05:39:17.209989+0000","last_unstale":"2023-12-04T05:39:18.518614+0000","last_undegraded":"2023-12-04T05:39:18.518614+0000","last_fullsized":"2023-12-04T05:39:18.518614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195553+0000","last_change":"2023-12-04T05:39:17.207109+0000","last_active":"2023-12-04T05:39:18.195553+0000","last_peered":"2023-12-04T05:39:18.195553+0000","last_clean":"2023-12-04T05:39:18.195553+0000","last_became_active":"2023-12-04T05:39:17.205722+0000","last_became_peered":"2023-12-04T05:39:17.205722+0000","last_unstale":"2023-12-04T05:39:18.195553+0000","last_undegraded":"2023-12-04T05:39:18.195553+0000","last_fullsized":"2023-12-04T05:39:18.195553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194533+0000","last_change":"2023-12-04T05:39:17.213549+0000","last_active":"2023-12-04T05:39:18.194533+0000","last_peered":"2023-12-04T05:39:18.194533+0000","last_clean":"2023-12-04T05:39:18.194533+0000","last_became_active":"2023-12-04T05:39:17.212758+0000","last_became_peered":"2023-12-04T05:39:17.212758+0000","last_unstale":"2023-12-04T05:39:18.194533+0000","last_undegraded":"2023-12-04T05:39:18.194533+0000","last_fullsized":"2023-12-04T05:39:18.194533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195513+0000","last_change":"2023-12-04T05:39:17.210092+0000","last_active":"2023-12-04T05:39:18.195513+0000","last_peered":"2023-12-04T05:39:18.195513+0000","last_clean":"2023-12-04T05:39:18.195513+0000","last_became_active":"2023-12-04T05:39:17.209828+0000","last_became_peered":"2023-12-04T05:39:17.209828+0000","last_unstale":"2023-12-04T05:39:18.195513+0000","last_undegraded":"2023-12-04T05:39:18.195513+0000","last_fullsized":"2023-12-04T05:39:18.195513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:01:33.532203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518749+0000","last_change":"2023-12-04T05:39:17.206756+0000","last_active":"2023-12-04T05:39:18.518749+0000","last_peered":"2023-12-04T05:39:18.518749+0000","last_clean":"2023-12-04T05:39:18.518749+0000","last_became_active":"2023-12-04T05:39:17.206499+0000","last_became_peered":"2023-12-04T05:39:17.206499+0000","last_unstale":"2023-12-04T05:39:18.518749+0000","last_undegraded":"2023-12-04T05:39:18.518749+0000","last_fullsized":"2023-12-04T05:39:18.518749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:33.381391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519868+0000","last_change":"2023-12-04T05:39:17.213693+0000","last_active":"2023-12-04T05:39:18.519868+0000","last_peered":"2023-12-04T05:39:18.519868+0000","last_clean":"2023-12-04T05:39:18.519868+0000","last_became_active":"2023-12-04T05:39:17.213404+0000","last_became_peered":"2023-12-04T05:39:17.213404+0000","last_unstale":"2023-12-04T05:39:18.519868+0000","last_undegraded":"2023-12-04T05:39:18.519868+0000","last_fullsized":"2023-12-04T05:39:18.519868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:27:18.604272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519451+0000","last_change":"2023-12-04T05:39:17.211735+0000","last_active":"2023-12-04T05:39:18.519451+0000","last_peered":"2023-12-04T05:39:18.519451+0000","last_clean":"2023-12-04T05:39:18.519451+0000","last_became_active":"2023-12-04T05:39:17.211450+0000","last_became_peered":"2023-12-04T05:39:17.211450+0000","last_unstale":"2023-12-04T05:39:18.519451+0000","last_undegraded":"2023-12-04T05:39:18.519451+0000","last_fullsized":"2023-12-04T05:39:18.519451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518422+0000","last_change":"2023-12-04T05:39:17.215901+0000","last_active":"2023-12-04T05:39:18.518422+0000","last_peered":"2023-12-04T05:39:18.518422+0000","last_clean":"2023-12-04T05:39:18.518422+0000","last_became_active":"2023-12-04T05:39:17.215395+0000","last_became_peered":"2023-12-04T05:39:17.215395+0000","last_unstale":"2023-12-04T05:39:18.518422+0000","last_undegraded":"2023-12-04T05:39:18.518422+0000","last_fullsized":"2023-12-04T05:39:18.518422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:01.587525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195031+0000","last_change":"2023-12-04T05:39:17.211134+0000","last_active":"2023-12-04T05:39:18.195031+0000","last_peered":"2023-12-04T05:39:18.195031+0000","last_clean":"2023-12-04T05:39:18.195031+0000","last_became_active":"2023-12-04T05:39:17.210734+0000","last_became_peered":"2023-12-04T05:39:17.210734+0000","last_unstale":"2023-12-04T05:39:18.195031+0000","last_undegraded":"2023-12-04T05:39:18.195031+0000","last_fullsized":"2023-12-04T05:39:18.195031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495425+0000","last_change":"2023-12-04T05:39:17.211310+0000","last_active":"2023-12-04T05:39:18.495425+0000","last_peered":"2023-12-04T05:39:18.495425+0000","last_clean":"2023-12-04T05:39:18.495425+0000","last_became_active":"2023-12-04T05:39:17.210967+0000","last_became_peered":"2023-12-04T05:39:17.210967+0000","last_unstale":"2023-12-04T05:39:18.495425+0000","last_undegraded":"2023-12-04T05:39:18.495425+0000","last_fullsized":"2023-12-04T05:39:18.495425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194980+0000","last_change":"2023-12-04T05:39:17.212493+0000","last_active":"2023-12-04T05:39:18.194980+0000","last_peered":"2023-12-04T05:39:18.194980+0000","last_clean":"2023-12-04T05:39:18.194980+0000","last_became_active":"2023-12-04T05:39:17.212240+0000","last_became_peered":"2023-12-04T05:39:17.212240+0000","last_unstale":"2023-12-04T05:39:18.194980+0000","last_undegraded":"2023-12-04T05:39:18.194980+0000","last_fullsized":"2023-12-04T05:39:18.194980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495743+0000","last_change":"2023-12-04T05:39:17.211088+0000","last_active":"2023-12-04T05:39:18.495743+0000","last_peered":"2023-12-04T05:39:18.495743+0000","last_clean":"2023-12-04T05:39:18.495743+0000","last_became_active":"2023-12-04T05:39:17.210779+0000","last_became_peered":"2023-12-04T05:39:17.210779+0000","last_unstale":"2023-12-04T05:39:18.495743+0000","last_undegraded":"2023-12-04T05:39:18.495743+0000","last_fullsized":"2023-12-04T05:39:18.495743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519676+0000","last_change":"2023-12-04T05:39:17.213404+0000","last_active":"2023-12-04T05:39:18.519676+0000","last_peered":"2023-12-04T05:39:18.519676+0000","last_clean":"2023-12-04T05:39:18.519676+0000","last_became_active":"2023-12-04T05:39:17.213153+0000","last_became_peered":"2023-12-04T05:39:17.213153+0000","last_unstale":"2023-12-04T05:39:18.519676+0000","last_undegraded":"2023-12-04T05:39:18.519676+0000","last_fullsized":"2023-12-04T05:39:18.519676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195771+0000","last_change":"2023-12-04T05:39:17.207997+0000","last_active":"2023-12-04T05:39:18.195771+0000","last_peered":"2023-12-04T05:39:18.195771+0000","last_clean":"2023-12-04T05:39:18.195771+0000","last_became_active":"2023-12-04T05:39:17.207537+0000","last_became_peered":"2023-12-04T05:39:17.207537+0000","last_unstale":"2023-12-04T05:39:18.195771+0000","last_undegraded":"2023-12-04T05:39:18.195771+0000","last_fullsized":"2023-12-04T05:39:18.195771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495833+0000","last_change":"2023-12-04T05:39:17.211215+0000","last_active":"2023-12-04T05:39:18.495833+0000","last_peered":"2023-12-04T05:39:18.495833+0000","last_clean":"2023-12-04T05:39:18.495833+0000","last_became_active":"2023-12-04T05:39:17.210930+0000","last_became_peered":"2023-12-04T05:39:17.210930+0000","last_unstale":"2023-12-04T05:39:18.495833+0000","last_undegraded":"2023-12-04T05:39:18.495833+0000","last_fullsized":"2023-12-04T05:39:18.495833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518602+0000","last_change":"2023-12-04T05:39:17.215388+0000","last_active":"2023-12-04T05:39:18.518602+0000","last_peered":"2023-12-04T05:39:18.518602+0000","last_clean":"2023-12-04T05:39:18.518602+0000","last_became_active":"2023-12-04T05:39:17.215082+0000","last_became_peered":"2023-12-04T05:39:17.215082+0000","last_unstale":"2023-12-04T05:39:18.518602+0000","last_undegraded":"2023-12-04T05:39:18.518602+0000","last_fullsized":"2023-12-04T05:39:18.518602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195233+0000","last_change":"2023-12-04T05:39:17.207162+0000","last_active":"2023-12-04T05:39:18.195233+0000","last_peered":"2023-12-04T05:39:18.195233+0000","last_clean":"2023-12-04T05:39:18.195233+0000","last_became_active":"2023-12-04T05:39:17.206737+0000","last_became_peered":"2023-12-04T05:39:17.206737+0000","last_unstale":"2023-12-04T05:39:18.195233+0000","last_undegraded":"2023-12-04T05:39:18.195233+0000","last_fullsized":"2023-12-04T05:39:18.195233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518154+0000","last_change":"2023-12-04T05:39:17.204908+0000","last_active":"2023-12-04T05:39:18.518154+0000","last_peered":"2023-12-04T05:39:18.518154+0000","last_clean":"2023-12-04T05:39:18.518154+0000","last_became_active":"2023-12-04T05:39:17.204656+0000","last_became_peered":"2023-12-04T05:39:17.204656+0000","last_unstale":"2023-12-04T05:39:18.518154+0000","last_undegraded":"2023-12-04T05:39:18.518154+0000","last_fullsized":"2023-12-04T05:39:18.518154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:33.059240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518987+0000","last_change":"2023-12-04T05:39:17.211113+0000","last_active":"2023-12-04T05:39:18.518987+0000","last_peered":"2023-12-04T05:39:18.518987+0000","last_clean":"2023-12-04T05:39:18.518987+0000","last_became_active":"2023-12-04T05:39:17.210626+0000","last_became_peered":"2023-12-04T05:39:17.210626+0000","last_unstale":"2023-12-04T05:39:18.518987+0000","last_undegraded":"2023-12-04T05:39:18.518987+0000","last_fullsized":"2023-12-04T05:39:18.518987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194380+0000","last_change":"2023-12-04T05:39:17.207261+0000","last_active":"2023-12-04T05:39:18.194380+0000","last_peered":"2023-12-04T05:39:18.194380+0000","last_clean":"2023-12-04T05:39:18.194380+0000","last_became_active":"2023-12-04T05:39:17.206957+0000","last_became_peered":"2023-12-04T05:39:17.206957+0000","last_unstale":"2023-12-04T05:39:18.194380+0000","last_undegraded":"2023-12-04T05:39:18.194380+0000","last_fullsized":"2023-12-04T05:39:18.194380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:38:49.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495637+0000","last_change":"2023-12-04T05:39:16.222472+0000","last_active":"2023-12-04T05:39:18.495637+0000","last_peered":"2023-12-04T05:39:18.495637+0000","last_clean":"2023-12-04T05:39:18.495637+0000","last_became_active":"2023-12-04T05:39:16.222136+0000","last_became_peered":"2023-12-04T05:39:16.222136+0000","last_unstale":"2023-12-04T05:39:18.495637+0000","last_undegraded":"2023-12-04T05:39:18.495637+0000","last_fullsized":"2023-12-04T05:39:18.495637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518714+0000","last_change":"2023-12-04T05:39:17.215145+0000","last_active":"2023-12-04T05:39:18.518714+0000","last_peered":"2023-12-04T05:39:18.518714+0000","last_clean":"2023-12-04T05:39:18.518714+0000","last_became_active":"2023-12-04T05:39:17.214877+0000","last_became_peered":"2023-12-04T05:39:17.214877+0000","last_unstale":"2023-12-04T05:39:18.518714+0000","last_undegraded":"2023-12-04T05:39:18.518714+0000","last_fullsized":"2023-12-04T05:39:18.518714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519318+0000","last_change":"2023-12-04T05:39:16.216663+0000","last_active":"2023-12-04T05:39:18.519318+0000","last_peered":"2023-12-04T05:39:18.519318+0000","last_clean":"2023-12-04T05:39:18.519318+0000","last_became_active":"2023-12-04T05:39:16.216408+0000","last_became_peered":"2023-12-04T05:39:16.216408+0000","last_unstale":"2023-12-04T05:39:18.519318+0000","last_undegraded":"2023-12-04T05:39:18.519318+0000","last_fullsized":"2023-12-04T05:39:18.519318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:01:24.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495399+0000","last_change":"2023-12-04T05:39:16.216780+0000","last_active":"2023-12-04T05:39:18.495399+0000","last_peered":"2023-12-04T05:39:18.495399+0000","last_clean":"2023-12-04T05:39:18.495399+0000","last_became_active":"2023-12-04T05:39:16.216486+0000","last_became_peered":"2023-12-04T05:39:16.216486+0000","last_unstale":"2023-12-04T05:39:18.495399+0000","last_undegraded":"2023-12-04T05:39:18.495399+0000","last_fullsized":"2023-12-04T05:39:18.495399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:00.582466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495379+0000","last_change":"2023-12-04T05:39:17.215850+0000","last_active":"2023-12-04T05:39:18.495379+0000","last_peered":"2023-12-04T05:39:18.495379+0000","last_clean":"2023-12-04T05:39:18.495379+0000","last_became_active":"2023-12-04T05:39:17.215557+0000","last_became_peered":"2023-12-04T05:39:17.215557+0000","last_unstale":"2023-12-04T05:39:18.495379+0000","last_undegraded":"2023-12-04T05:39:18.495379+0000","last_fullsized":"2023-12-04T05:39:18.495379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194854+0000","last_change":"2023-12-04T05:39:16.213564+0000","last_active":"2023-12-04T05:39:18.194854+0000","last_peered":"2023-12-04T05:39:18.194854+0000","last_clean":"2023-12-04T05:39:18.194854+0000","last_became_active":"2023-12-04T05:39:16.213288+0000","last_became_peered":"2023-12-04T05:39:16.213288+0000","last_unstale":"2023-12-04T05:39:18.194854+0000","last_undegraded":"2023-12-04T05:39:18.194854+0000","last_fullsized":"2023-12-04T05:39:18.194854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195514+0000","last_change":"2023-12-04T05:39:17.208143+0000","last_active":"2023-12-04T05:39:18.195514+0000","last_peered":"2023-12-04T05:39:18.195514+0000","last_clean":"2023-12-04T05:39:18.195514+0000","last_became_active":"2023-12-04T05:39:17.207883+0000","last_became_peered":"2023-12-04T05:39:17.207883+0000","last_unstale":"2023-12-04T05:39:18.195514+0000","last_undegraded":"2023-12-04T05:39:18.195514+0000","last_fullsized":"2023-12-04T05:39:18.195514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519244+0000","last_change":"2023-12-04T05:39:16.217144+0000","last_active":"2023-12-04T05:39:18.519244+0000","last_peered":"2023-12-04T05:39:18.519244+0000","last_clean":"2023-12-04T05:39:18.519244+0000","last_became_active":"2023-12-04T05:39:16.216759+0000","last_became_peered":"2023-12-04T05:39:16.216759+0000","last_unstale":"2023-12-04T05:39:18.519244+0000","last_undegraded":"2023-12-04T05:39:18.519244+0000","last_fullsized":"2023-12-04T05:39:18.519244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:22:44.119471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195253+0000","last_change":"2023-12-04T05:39:17.211977+0000","last_active":"2023-12-04T05:39:18.195253+0000","last_peered":"2023-12-04T05:39:18.195253+0000","last_clean":"2023-12-04T05:39:18.195253+0000","last_became_active":"2023-12-04T05:39:17.211707+0000","last_became_peered":"2023-12-04T05:39:17.211707+0000","last_unstale":"2023-12-04T05:39:18.195253+0000","last_undegraded":"2023-12-04T05:39:18.195253+0000","last_fullsized":"2023-12-04T05:39:18.195253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195229+0000","last_change":"2023-12-04T05:39:16.212865+0000","last_active":"2023-12-04T05:39:18.195229+0000","last_peered":"2023-12-04T05:39:18.195229+0000","last_clean":"2023-12-04T05:39:18.195229+0000","last_became_active":"2023-12-04T05:39:16.212574+0000","last_became_peered":"2023-12-04T05:39:16.212574+0000","last_unstale":"2023-12-04T05:39:18.195229+0000","last_undegraded":"2023-12-04T05:39:18.195229+0000","last_fullsized":"2023-12-04T05:39:18.195229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:17.975393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518554+0000","last_change":"2023-12-04T05:39:17.210657+0000","last_active":"2023-12-04T05:39:18.518554+0000","last_peered":"2023-12-04T05:39:18.518554+0000","last_clean":"2023-12-04T05:39:18.518554+0000","last_became_active":"2023-12-04T05:39:17.210197+0000","last_became_peered":"2023-12-04T05:39:17.210197+0000","last_unstale":"2023-12-04T05:39:18.518554+0000","last_undegraded":"2023-12-04T05:39:18.518554+0000","last_fullsized":"2023-12-04T05:39:18.518554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195730+0000","last_change":"2023-12-04T05:39:16.219612+0000","last_active":"2023-12-04T05:39:18.195730+0000","last_peered":"2023-12-04T05:39:18.195730+0000","last_clean":"2023-12-04T05:39:18.195730+0000","last_became_active":"2023-12-04T05:39:16.219305+0000","last_became_peered":"2023-12-04T05:39:16.219305+0000","last_unstale":"2023-12-04T05:39:18.195730+0000","last_undegraded":"2023-12-04T05:39:18.195730+0000","last_fullsized":"2023-12-04T05:39:18.195730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194796+0000","last_change":"2023-12-04T05:39:17.202594+0000","last_active":"2023-12-04T05:39:18.194796+0000","last_peered":"2023-12-04T05:39:18.194796+0000","last_clean":"2023-12-04T05:39:18.194796+0000","last_became_active":"2023-12-04T05:39:17.202079+0000","last_became_peered":"2023-12-04T05:39:17.202079+0000","last_unstale":"2023-12-04T05:39:18.194796+0000","last_undegraded":"2023-12-04T05:39:18.194796+0000","last_fullsized":"2023-12-04T05:39:18.194796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:33.059240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519559+0000","last_change":"2023-12-04T05:39:16.212413+0000","last_active":"2023-12-04T05:39:18.519559+0000","last_peered":"2023-12-04T05:39:18.519559+0000","last_clean":"2023-12-04T05:39:18.519559+0000","last_became_active":"2023-12-04T05:39:16.212127+0000","last_became_peered":"2023-12-04T05:39:16.212127+0000","last_unstale":"2023-12-04T05:39:18.519559+0000","last_undegraded":"2023-12-04T05:39:18.519559+0000","last_fullsized":"2023-12-04T05:39:18.519559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519613+0000","last_change":"2023-12-04T05:39:17.208325+0000","last_active":"2023-12-04T05:39:18.519613+0000","last_peered":"2023-12-04T05:39:18.519613+0000","last_clean":"2023-12-04T05:39:18.519613+0000","last_became_active":"2023-12-04T05:39:17.208060+0000","last_became_peered":"2023-12-04T05:39:17.208060+0000","last_unstale":"2023-12-04T05:39:18.519613+0000","last_undegraded":"2023-12-04T05:39:18.519613+0000","last_fullsized":"2023-12-04T05:39:18.519613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:56:49.040036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518855+0000","last_change":"2023-12-04T05:39:16.217018+0000","last_active":"2023-12-04T05:39:18.518855+0000","last_peered":"2023-12-04T05:39:18.518855+0000","last_clean":"2023-12-04T05:39:18.518855+0000","last_became_active":"2023-12-04T05:39:16.216772+0000","last_became_peered":"2023-12-04T05:39:16.216772+0000","last_unstale":"2023-12-04T05:39:18.518855+0000","last_undegraded":"2023-12-04T05:39:18.518855+0000","last_fullsized":"2023-12-04T05:39:18.518855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:32.054181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518705+0000","last_change":"2023-12-04T05:39:17.213778+0000","last_active":"2023-12-04T05:39:18.518705+0000","last_peered":"2023-12-04T05:39:18.518705+0000","last_clean":"2023-12-04T05:39:18.518705+0000","last_became_active":"2023-12-04T05:39:17.213520+0000","last_became_peered":"2023-12-04T05:39:17.213520+0000","last_unstale":"2023-12-04T05:39:18.518705+0000","last_undegraded":"2023-12-04T05:39:18.518705+0000","last_fullsized":"2023-12-04T05:39:18.518705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195291+0000","last_change":"2023-12-04T05:39:16.212009+0000","last_active":"2023-12-04T05:39:18.195291+0000","last_peered":"2023-12-04T05:39:18.195291+0000","last_clean":"2023-12-04T05:39:18.195291+0000","last_became_active":"2023-12-04T05:39:16.211611+0000","last_became_peered":"2023-12-04T05:39:16.211611+0000","last_unstale":"2023-12-04T05:39:18.195291+0000","last_undegraded":"2023-12-04T05:39:18.195291+0000","last_fullsized":"2023-12-04T05:39:18.195291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:24.393125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519548+0000","last_change":"2023-12-04T05:39:17.210008+0000","last_active":"2023-12-04T05:39:18.519548+0000","last_peered":"2023-12-04T05:39:18.519548+0000","last_clean":"2023-12-04T05:39:18.519548+0000","last_became_active":"2023-12-04T05:39:17.209767+0000","last_became_peered":"2023-12-04T05:39:17.209767+0000","last_unstale":"2023-12-04T05:39:18.519548+0000","last_undegraded":"2023-12-04T05:39:18.519548+0000","last_fullsized":"2023-12-04T05:39:18.519548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:35:27.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519053+0000","last_change":"2023-12-04T05:39:16.219504+0000","last_active":"2023-12-04T05:39:18.519053+0000","last_peered":"2023-12-04T05:39:18.519053+0000","last_clean":"2023-12-04T05:39:18.519053+0000","last_became_active":"2023-12-04T05:39:16.219241+0000","last_became_peered":"2023-12-04T05:39:16.219241+0000","last_unstale":"2023-12-04T05:39:18.519053+0000","last_undegraded":"2023-12-04T05:39:18.519053+0000","last_fullsized":"2023-12-04T05:39:18.519053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.913440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519043+0000","last_change":"2023-12-04T05:39:17.210165+0000","last_active":"2023-12-04T05:39:18.519043+0000","last_peered":"2023-12-04T05:39:18.519043+0000","last_clean":"2023-12-04T05:39:18.519043+0000","last_became_active":"2023-12-04T05:39:17.209917+0000","last_became_peered":"2023-12-04T05:39:17.209917+0000","last_unstale":"2023-12-04T05:39:18.519043+0000","last_undegraded":"2023-12-04T05:39:18.519043+0000","last_fullsized":"2023-12-04T05:39:18.519043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518077+0000","last_change":"2023-12-04T05:39:16.216340+0000","last_active":"2023-12-04T05:39:18.518077+0000","last_peered":"2023-12-04T05:39:18.518077+0000","last_clean":"2023-12-04T05:39:18.518077+0000","last_became_active":"2023-12-04T05:39:16.216034+0000","last_became_peered":"2023-12-04T05:39:16.216034+0000","last_unstale":"2023-12-04T05:39:18.518077+0000","last_undegraded":"2023-12-04T05:39:18.518077+0000","last_fullsized":"2023-12-04T05:39:18.518077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.913440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519028+0000","last_change":"2023-12-04T05:39:17.216087+0000","last_active":"2023-12-04T05:39:18.519028+0000","last_peered":"2023-12-04T05:39:18.519028+0000","last_clean":"2023-12-04T05:39:18.519028+0000","last_became_active":"2023-12-04T05:39:17.215420+0000","last_became_peered":"2023-12-04T05:39:17.215420+0000","last_unstale":"2023-12-04T05:39:18.519028+0000","last_undegraded":"2023-12-04T05:39:18.519028+0000","last_fullsized":"2023-12-04T05:39:18.519028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194623+0000","last_change":"2023-12-04T05:39:16.218872+0000","last_active":"2023-12-04T05:39:18.194623+0000","last_peered":"2023-12-04T05:39:18.194623+0000","last_clean":"2023-12-04T05:39:18.194623+0000","last_became_active":"2023-12-04T05:39:16.218609+0000","last_became_peered":"2023-12-04T05:39:16.218609+0000","last_unstale":"2023-12-04T05:39:18.194623+0000","last_undegraded":"2023-12-04T05:39:18.194623+0000","last_fullsized":"2023-12-04T05:39:18.194623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T05:42:48.541211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519439+0000","last_change":"2023-12-04T05:39:17.209465+0000","last_active":"2023-12-04T05:39:18.519439+0000","last_peered":"2023-12-04T05:39:18.519439+0000","last_clean":"2023-12-04T05:39:18.519439+0000","last_became_active":"2023-12-04T05:39:17.209210+0000","last_became_peered":"2023-12-04T05:39:17.209210+0000","last_unstale":"2023-12-04T05:39:18.519439+0000","last_undegraded":"2023-12-04T05:39:18.519439+0000","last_fullsized":"2023-12-04T05:39:18.519439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518796+0000","last_change":"2023-12-04T05:39:16.212731+0000","last_active":"2023-12-04T05:39:18.518796+0000","last_peered":"2023-12-04T05:39:18.518796+0000","last_clean":"2023-12-04T05:39:18.518796+0000","last_became_active":"2023-12-04T05:39:16.212398+0000","last_became_peered":"2023-12-04T05:39:16.212398+0000","last_unstale":"2023-12-04T05:39:18.518796+0000","last_undegraded":"2023-12-04T05:39:18.518796+0000","last_fullsized":"2023-12-04T05:39:18.518796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:13.519031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519792+0000","last_change":"2023-12-04T05:39:17.215878+0000","last_active":"2023-12-04T05:39:18.519792+0000","last_peered":"2023-12-04T05:39:18.519792+0000","last_clean":"2023-12-04T05:39:18.519792+0000","last_became_active":"2023-12-04T05:39:17.215600+0000","last_became_peered":"2023-12-04T05:39:17.215600+0000","last_unstale":"2023-12-04T05:39:18.519792+0000","last_undegraded":"2023-12-04T05:39:18.519792+0000","last_fullsized":"2023-12-04T05:39:18.519792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:21:50.865362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195801+0000","last_change":"2023-12-04T05:39:16.219401+0000","last_active":"2023-12-04T05:39:18.195801+0000","last_peered":"2023-12-04T05:39:18.195801+0000","last_clean":"2023-12-04T05:39:18.195801+0000","last_became_active":"2023-12-04T05:39:16.219096+0000","last_became_peered":"2023-12-04T05:39:16.219096+0000","last_unstale":"2023-12-04T05:39:18.195801+0000","last_undegraded":"2023-12-04T05:39:18.195801+0000","last_fullsized":"2023-12-04T05:39:18.195801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195394+0000","last_change":"2023-12-04T05:39:17.208978+0000","last_active":"2023-12-04T05:39:18.195394+0000","last_peered":"2023-12-04T05:39:18.195394+0000","last_clean":"2023-12-04T05:39:18.195394+0000","last_became_active":"2023-12-04T05:39:17.208163+0000","last_became_peered":"2023-12-04T05:39:17.208163+0000","last_unstale":"2023-12-04T05:39:18.195394+0000","last_undegraded":"2023-12-04T05:39:18.195394+0000","last_fullsized":"2023-12-04T05:39:18.195394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:35:27.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518395+0000","last_change":"2023-12-04T05:39:16.216610+0000","last_active":"2023-12-04T05:39:18.518395+0000","last_peered":"2023-12-04T05:39:18.518395+0000","last_clean":"2023-12-04T05:39:18.518395+0000","last_became_active":"2023-12-04T05:39:16.216343+0000","last_became_peered":"2023-12-04T05:39:16.216343+0000","last_unstale":"2023-12-04T05:39:18.518395+0000","last_undegraded":"2023-12-04T05:39:18.518395+0000","last_fullsized":"2023-12-04T05:39:18.518395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:06:28.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519731+0000","last_change":"2023-12-04T05:39:17.206092+0000","last_active":"2023-12-04T05:39:18.519731+0000","last_peered":"2023-12-04T05:39:18.519731+0000","last_clean":"2023-12-04T05:39:18.519731+0000","last_became_active":"2023-12-04T05:39:17.205430+0000","last_became_peered":"2023-12-04T05:39:17.205430+0000","last_unstale":"2023-12-04T05:39:18.519731+0000","last_undegraded":"2023-12-04T05:39:18.519731+0000","last_fullsized":"2023-12-04T05:39:18.519731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:28:25.883452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-04T05:39:17.200250+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:39:17.200250+0000","last_peered":"2023-12-04T05:39:17.200250+0000","last_clean":"2023-12-04T05:39:17.200250+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:39:17.200250+0000","last_undegraded":"2023-12-04T05:39:17.200250+0000","last_fullsized":"2023-12-04T05:39:17.200250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194896+0000","last_change":"2023-12-04T05:39:16.211697+0000","last_active":"2023-12-04T05:39:18.194896+0000","last_peered":"2023-12-04T05:39:18.194896+0000","last_clean":"2023-12-04T05:39:18.194896+0000","last_became_active":"2023-12-04T05:39:16.211421+0000","last_became_peered":"2023-12-04T05:39:16.211421+0000","last_unstale":"2023-12-04T05:39:18.194896+0000","last_undegraded":"2023-12-04T05:39:18.194896+0000","last_fullsized":"2023-12-04T05:39:18.194896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:05.599778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495509+0000","last_change":"2023-12-04T05:39:17.212309+0000","last_active":"2023-12-04T05:39:18.495509+0000","last_peered":"2023-12-04T05:39:18.495509+0000","last_clean":"2023-12-04T05:39:18.495509+0000","last_became_active":"2023-12-04T05:39:17.212069+0000","last_became_peered":"2023-12-04T05:39:17.212069+0000","last_unstale":"2023-12-04T05:39:18.495509+0000","last_undegraded":"2023-12-04T05:39:18.495509+0000","last_fullsized":"2023-12-04T05:39:18.495509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518862+0000","last_change":"2023-12-04T05:39:16.222456+0000","last_active":"2023-12-04T05:39:18.518862+0000","last_peered":"2023-12-04T05:39:18.518862+0000","last_clean":"2023-12-04T05:39:18.518862+0000","last_became_active":"2023-12-04T05:39:16.222007+0000","last_became_peered":"2023-12-04T05:39:16.222007+0000","last_unstale":"2023-12-04T05:39:18.518862+0000","last_undegraded":"2023-12-04T05:39:18.518862+0000","last_fullsized":"2023-12-04T05:39:18.518862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518357+0000","last_change":"2023-12-04T05:39:17.209347+0000","last_active":"2023-12-04T05:39:18.518357+0000","last_peered":"2023-12-04T05:39:18.518357+0000","last_clean":"2023-12-04T05:39:18.518357+0000","last_became_active":"2023-12-04T05:39:17.209044+0000","last_became_peered":"2023-12-04T05:39:17.209044+0000","last_unstale":"2023-12-04T05:39:18.518357+0000","last_undegraded":"2023-12-04T05:39:18.518357+0000","last_fullsized":"2023-12-04T05:39:18.518357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:40.539141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518478+0000","last_change":"2023-12-04T05:39:16.213448+0000","last_active":"2023-12-04T05:39:18.518478+0000","last_peered":"2023-12-04T05:39:18.518478+0000","last_clean":"2023-12-04T05:39:18.518478+0000","last_became_active":"2023-12-04T05:39:16.212426+0000","last_became_peered":"2023-12-04T05:39:16.212426+0000","last_unstale":"2023-12-04T05:39:18.518478+0000","last_undegraded":"2023-12-04T05:39:18.518478+0000","last_fullsized":"2023-12-04T05:39:18.518478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:00:49.392004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195538+0000","last_change":"2023-12-04T05:39:17.206807+0000","last_active":"2023-12-04T05:39:18.195538+0000","last_peered":"2023-12-04T05:39:18.195538+0000","last_clean":"2023-12-04T05:39:18.195538+0000","last_became_active":"2023-12-04T05:39:17.206502+0000","last_became_peered":"2023-12-04T05:39:17.206502+0000","last_unstale":"2023-12-04T05:39:18.195538+0000","last_undegraded":"2023-12-04T05:39:18.195538+0000","last_fullsized":"2023-12-04T05:39:18.195538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195374+0000","last_change":"2023-12-04T05:39:16.216126+0000","last_active":"2023-12-04T05:39:18.195374+0000","last_peered":"2023-12-04T05:39:18.195374+0000","last_clean":"2023-12-04T05:39:18.195374+0000","last_became_active":"2023-12-04T05:39:16.215729+0000","last_became_peered":"2023-12-04T05:39:16.215729+0000","last_unstale":"2023-12-04T05:39:18.195374+0000","last_undegraded":"2023-12-04T05:39:18.195374+0000","last_fullsized":"2023-12-04T05:39:18.195374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:18.392982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194711+0000","last_change":"2023-12-04T05:39:17.215887+0000","last_active":"2023-12-04T05:39:18.194711+0000","last_peered":"2023-12-04T05:39:18.194711+0000","last_clean":"2023-12-04T05:39:18.194711+0000","last_became_active":"2023-12-04T05:39:17.215638+0000","last_became_peered":"2023-12-04T05:39:17.215638+0000","last_unstale":"2023-12-04T05:39:18.194711+0000","last_undegraded":"2023-12-04T05:39:18.194711+0000","last_fullsized":"2023-12-04T05:39:18.194711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:01:25.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195699+0000","last_change":"2023-12-04T05:39:16.222396+0000","last_active":"2023-12-04T05:39:18.195699+0000","last_peered":"2023-12-04T05:39:18.195699+0000","last_clean":"2023-12-04T05:39:18.195699+0000","last_became_active":"2023-12-04T05:39:16.222118+0000","last_became_peered":"2023-12-04T05:39:16.222118+0000","last_unstale":"2023-12-04T05:39:18.195699+0000","last_undegraded":"2023-12-04T05:39:18.195699+0000","last_fullsized":"2023-12-04T05:39:18.195699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195595+0000","last_change":"2023-12-04T05:39:17.211083+0000","last_active":"2023-12-04T05:39:18.195595+0000","last_peered":"2023-12-04T05:39:18.195595+0000","last_clean":"2023-12-04T05:39:18.195595+0000","last_became_active":"2023-12-04T05:39:17.210751+0000","last_became_peered":"2023-12-04T05:39:17.210751+0000","last_unstale":"2023-12-04T05:39:18.195595+0000","last_undegraded":"2023-12-04T05:39:18.195595+0000","last_fullsized":"2023-12-04T05:39:18.195595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:28:25.883452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519823+0000","last_change":"2023-12-04T05:39:16.222777+0000","last_active":"2023-12-04T05:39:18.519823+0000","last_peered":"2023-12-04T05:39:18.519823+0000","last_clean":"2023-12-04T05:39:18.519823+0000","last_became_active":"2023-12-04T05:39:16.222487+0000","last_became_peered":"2023-12-04T05:39:16.222487+0000","last_unstale":"2023-12-04T05:39:18.519823+0000","last_undegraded":"2023-12-04T05:39:18.519823+0000","last_fullsized":"2023-12-04T05:39:18.519823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:20.634044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195752+0000","last_change":"2023-12-04T05:39:17.206784+0000","last_active":"2023-12-04T05:39:18.195752+0000","last_peered":"2023-12-04T05:39:18.195752+0000","last_clean":"2023-12-04T05:39:18.195752+0000","last_became_active":"2023-12-04T05:39:17.206468+0000","last_became_peered":"2023-12-04T05:39:17.206468+0000","last_unstale":"2023-12-04T05:39:18.195752+0000","last_undegraded":"2023-12-04T05:39:18.195752+0000","last_fullsized":"2023-12-04T05:39:18.195752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:19.490804+0000","last_change":"2023-12-04T05:39:16.219828+0000","last_active":"2023-12-04T05:39:19.490804+0000","last_peered":"2023-12-04T05:39:19.490804+0000","last_clean":"2023-12-04T05:39:19.490804+0000","last_became_active":"2023-12-04T05:39:16.219564+0000","last_became_peered":"2023-12-04T05:39:16.219564+0000","last_unstale":"2023-12-04T05:39:19.490804+0000","last_undegraded":"2023-12-04T05:39:19.490804+0000","last_fullsized":"2023-12-04T05:39:19.490804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:00.582466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495806+0000","last_change":"2023-12-04T05:39:17.213462+0000","last_active":"2023-12-04T05:39:18.495806+0000","last_peered":"2023-12-04T05:39:18.495806+0000","last_clean":"2023-12-04T05:39:18.495806+0000","last_became_active":"2023-12-04T05:39:17.213163+0000","last_became_peered":"2023-12-04T05:39:17.213163+0000","last_unstale":"2023-12-04T05:39:18.495806+0000","last_undegraded":"2023-12-04T05:39:18.495806+0000","last_fullsized":"2023-12-04T05:39:18.495806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519346+0000","last_change":"2023-12-04T05:39:16.225197+0000","last_active":"2023-12-04T05:39:18.519346+0000","last_peered":"2023-12-04T05:39:18.519346+0000","last_clean":"2023-12-04T05:39:18.519346+0000","last_became_active":"2023-12-04T05:39:16.224896+0000","last_became_peered":"2023-12-04T05:39:16.224896+0000","last_unstale":"2023-12-04T05:39:18.519346+0000","last_undegraded":"2023-12-04T05:39:18.519346+0000","last_fullsized":"2023-12-04T05:39:18.519346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518458+0000","last_change":"2023-12-04T05:39:17.211876+0000","last_active":"2023-12-04T05:39:18.518458+0000","last_peered":"2023-12-04T05:39:18.518458+0000","last_clean":"2023-12-04T05:39:18.518458+0000","last_became_active":"2023-12-04T05:39:17.211580+0000","last_became_peered":"2023-12-04T05:39:17.211580+0000","last_unstale":"2023-12-04T05:39:18.518458+0000","last_undegraded":"2023-12-04T05:39:18.518458+0000","last_fullsized":"2023-12-04T05:39:18.518458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518857+0000","last_change":"2023-12-04T05:39:16.222816+0000","last_active":"2023-12-04T05:39:18.518857+0000","last_peered":"2023-12-04T05:39:18.518857+0000","last_clean":"2023-12-04T05:39:18.518857+0000","last_became_active":"2023-12-04T05:39:16.222453+0000","last_became_peered":"2023-12-04T05:39:16.222453+0000","last_unstale":"2023-12-04T05:39:18.518857+0000","last_undegraded":"2023-12-04T05:39:18.518857+0000","last_fullsized":"2023-12-04T05:39:18.518857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194429+0000","last_change":"2023-12-04T05:39:17.204065+0000","last_active":"2023-12-04T05:39:18.194429+0000","last_peered":"2023-12-04T05:39:18.194429+0000","last_clean":"2023-12-04T05:39:18.194429+0000","last_became_active":"2023-12-04T05:39:17.203770+0000","last_became_peered":"2023-12-04T05:39:17.203770+0000","last_unstale":"2023-12-04T05:39:18.194429+0000","last_undegraded":"2023-12-04T05:39:18.194429+0000","last_fullsized":"2023-12-04T05:39:18.194429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:00:50.397064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195575+0000","last_change":"2023-12-04T05:39:16.223148+0000","last_active":"2023-12-04T05:39:18.195575+0000","last_peered":"2023-12-04T05:39:18.195575+0000","last_clean":"2023-12-04T05:39:18.195575+0000","last_became_active":"2023-12-04T05:39:16.222884+0000","last_became_peered":"2023-12-04T05:39:16.222884+0000","last_unstale":"2023-12-04T05:39:18.195575+0000","last_undegraded":"2023-12-04T05:39:18.195575+0000","last_fullsized":"2023-12-04T05:39:18.195575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:32.054181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495446+0000","last_change":"2023-12-04T05:39:17.207953+0000","last_active":"2023-12-04T05:39:18.495446+0000","last_peered":"2023-12-04T05:39:18.495446+0000","last_clean":"2023-12-04T05:39:18.495446+0000","last_became_active":"2023-12-04T05:39:17.207612+0000","last_became_peered":"2023-12-04T05:39:17.207612+0000","last_unstale":"2023-12-04T05:39:18.495446+0000","last_undegraded":"2023-12-04T05:39:18.495446+0000","last_fullsized":"2023-12-04T05:39:18.495446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195660+0000","last_change":"2023-12-04T05:39:16.216886+0000","last_active":"2023-12-04T05:39:18.195660+0000","last_peered":"2023-12-04T05:39:18.195660+0000","last_clean":"2023-12-04T05:39:18.195660+0000","last_became_active":"2023-12-04T05:39:16.215666+0000","last_became_peered":"2023-12-04T05:39:16.215666+0000","last_unstale":"2023-12-04T05:39:18.195660+0000","last_undegraded":"2023-12-04T05:39:18.195660+0000","last_fullsized":"2023-12-04T05:39:18.195660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495633+0000","last_change":"2023-12-04T05:39:17.216210+0000","last_active":"2023-12-04T05:39:18.495633+0000","last_peered":"2023-12-04T05:39:18.495633+0000","last_clean":"2023-12-04T05:39:18.495633+0000","last_became_active":"2023-12-04T05:39:17.215926+0000","last_became_peered":"2023-12-04T05:39:17.215926+0000","last_unstale":"2023-12-04T05:39:18.495633+0000","last_undegraded":"2023-12-04T05:39:18.495633+0000","last_fullsized":"2023-12-04T05:39:18.495633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:53.681504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519432+0000","last_change":"2023-12-04T05:39:16.222353+0000","last_active":"2023-12-04T05:39:18.519432+0000","last_peered":"2023-12-04T05:39:18.519432+0000","last_clean":"2023-12-04T05:39:18.519432+0000","last_became_active":"2023-12-04T05:39:16.221956+0000","last_became_peered":"2023-12-04T05:39:16.221956+0000","last_unstale":"2023-12-04T05:39:18.519432+0000","last_undegraded":"2023-12-04T05:39:18.519432+0000","last_fullsized":"2023-12-04T05:39:18.519432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519216+0000","last_change":"2023-12-04T05:39:17.210884+0000","last_active":"2023-12-04T05:39:18.519216+0000","last_peered":"2023-12-04T05:39:18.519216+0000","last_clean":"2023-12-04T05:39:18.519216+0000","last_became_active":"2023-12-04T05:39:17.210601+0000","last_became_peered":"2023-12-04T05:39:17.210601+0000","last_unstale":"2023-12-04T05:39:18.519216+0000","last_undegraded":"2023-12-04T05:39:18.519216+0000","last_fullsized":"2023-12-04T05:39:18.519216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195487+0000","last_change":"2023-12-04T05:39:17.204261+0000","last_active":"2023-12-04T05:39:18.195487+0000","last_peered":"2023-12-04T05:39:18.195487+0000","last_clean":"2023-12-04T05:39:18.195487+0000","last_became_active":"2023-12-04T05:39:17.203985+0000","last_became_peered":"2023-12-04T05:39:17.203985+0000","last_unstale":"2023-12-04T05:39:18.195487+0000","last_undegraded":"2023-12-04T05:39:18.195487+0000","last_fullsized":"2023-12-04T05:39:18.195487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:56:49.040036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519438+0000","last_change":"2023-12-04T05:39:16.222742+0000","last_active":"2023-12-04T05:39:18.519438+0000","last_peered":"2023-12-04T05:39:18.519438+0000","last_clean":"2023-12-04T05:39:18.519438+0000","last_became_active":"2023-12-04T05:39:16.222073+0000","last_became_peered":"2023-12-04T05:39:16.222073+0000","last_unstale":"2023-12-04T05:39:18.519438+0000","last_undegraded":"2023-12-04T05:39:18.519438+0000","last_fullsized":"2023-12-04T05:39:18.519438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195197+0000","last_change":"2023-12-04T05:39:17.204727+0000","last_active":"2023-12-04T05:39:18.195197+0000","last_peered":"2023-12-04T05:39:18.195197+0000","last_clean":"2023-12-04T05:39:18.195197+0000","last_became_active":"2023-12-04T05:39:17.203997+0000","last_became_peered":"2023-12-04T05:39:17.203997+0000","last_unstale":"2023-12-04T05:39:18.195197+0000","last_undegraded":"2023-12-04T05:39:18.195197+0000","last_fullsized":"2023-12-04T05:39:18.195197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519715+0000","last_change":"2023-12-04T05:39:16.223432+0000","last_active":"2023-12-04T05:39:18.519715+0000","last_peered":"2023-12-04T05:39:18.519715+0000","last_clean":"2023-12-04T05:39:18.519715+0000","last_became_active":"2023-12-04T05:39:16.222966+0000","last_became_peered":"2023-12-04T05:39:16.222966+0000","last_unstale":"2023-12-04T05:39:18.519715+0000","last_undegraded":"2023-12-04T05:39:18.519715+0000","last_fullsized":"2023-12-04T05:39:18.519715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:24.393125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518460+0000","last_change":"2023-12-04T05:39:17.214833+0000","last_active":"2023-12-04T05:39:18.518460+0000","last_peered":"2023-12-04T05:39:18.518460+0000","last_clean":"2023-12-04T05:39:18.518460+0000","last_became_active":"2023-12-04T05:39:17.214570+0000","last_became_peered":"2023-12-04T05:39:17.214570+0000","last_unstale":"2023-12-04T05:39:18.518460+0000","last_undegraded":"2023-12-04T05:39:18.518460+0000","last_fullsized":"2023-12-04T05:39:18.518460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:24.110066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495773+0000","last_change":"2023-12-04T05:39:16.218785+0000","last_active":"2023-12-04T05:39:18.495773+0000","last_peered":"2023-12-04T05:39:18.495773+0000","last_clean":"2023-12-04T05:39:18.495773+0000","last_became_active":"2023-12-04T05:39:16.218480+0000","last_became_peered":"2023-12-04T05:39:16.218480+0000","last_unstale":"2023-12-04T05:39:18.495773+0000","last_undegraded":"2023-12-04T05:39:18.495773+0000","last_fullsized":"2023-12-04T05:39:18.495773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495766+0000","last_change":"2023-12-04T05:39:17.208223+0000","last_active":"2023-12-04T05:39:18.495766+0000","last_peered":"2023-12-04T05:39:18.495766+0000","last_clean":"2023-12-04T05:39:18.495766+0000","last_became_active":"2023-12-04T05:39:17.207977+0000","last_became_peered":"2023-12-04T05:39:17.207977+0000","last_unstale":"2023-12-04T05:39:18.495766+0000","last_undegraded":"2023-12-04T05:39:18.495766+0000","last_fullsized":"2023-12-04T05:39:18.495766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518609+0000","last_change":"2023-12-04T05:39:16.222709+0000","last_active":"2023-12-04T05:39:18.518609+0000","last_peered":"2023-12-04T05:39:18.518609+0000","last_clean":"2023-12-04T05:39:18.518609+0000","last_became_active":"2023-12-04T05:39:16.222155+0000","last_became_peered":"2023-12-04T05:39:16.222155+0000","last_unstale":"2023-12-04T05:39:18.518609+0000","last_undegraded":"2023-12-04T05:39:18.518609+0000","last_fullsized":"2023-12-04T05:39:18.518609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.613}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.04}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.193}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.219}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:39:24.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:39:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.544+0000 7fbf3fb05640 1 Processor -- start 2023-12-04T05:39:24.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.544+0000 7fbf3fb05640 1 -- start start 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3fb05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf3806c960 0x7fbf3815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3fb05640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf38140200 0x7fbf3815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3fb05640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf380aea20 con 0x7fbf3813cee0 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3fb05640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3813d2c0 con 0x7fbf38140200 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3fb05640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3815b3a0 con 0x7fbf3806c960 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf38140200 0x7fbf3815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf38140200 0x7fbf3815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59068/0 (socket says 172.21.15.111:59068) 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 -- 172.21.15.111:0/2998281257 learned_addr learned my addr 172.21.15.111:0/2998281257 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3d0a0640 1 --2- 172.21.15.111:0/2998281257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf3806c960 0x7fbf3815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 -- 172.21.15.111:0/2998281257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf3806c960 msgr2=0x7fbf3815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 --2- 172.21.15.111:0/2998281257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf3806c960 0x7fbf3815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 -- 172.21.15.111:0/2998281257 >> v1:172.21.15.192:6789/0 conn(0x7fbf3813cee0 legacy=0x7fbf3806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.545+0000 7fbf3e0a2640 1 -- 172.21.15.111:0/2998281257 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3815b0c0 con 0x7fbf38140200 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.546+0000 7fbf3e0a2640 1 --2- 172.21.15.111:0/2998281257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf38140200 0x7fbf3815a9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009e50 tx=0x7fbf3002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b6b0acc33b04b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.546+0000 7fbf3c89f640 1 -- 172.21.15.111:0/2998281257 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3002c9e0 con 0x7fbf38140200 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.546+0000 7fbf3c89f640 1 -- 172.21.15.111:0/2998281257 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3002cb40 con 0x7fbf38140200 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.546+0000 7fbf3c89f640 1 -- 172.21.15.111:0/2998281257 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300343f0 con 0x7fbf38140200 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.546+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2998281257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf38140200 msgr2=0x7fbf3815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.546+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2998281257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf38140200 0x7fbf3815a9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009e50 tx=0x7fbf3002be40 comp rx=0 tx=0).stop 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.547+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2998281257 shutdown_connections 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.547+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2998281257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf38140200 0x7fbf3815a9d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.547+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2998281257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf3806c960 0x7fbf3815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.547+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2998281257 >> 172.21.15.111:0/2998281257 conn(0x7fbf3806a4c0 msgr2=0x7fbf3806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.547+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2998281257 shutdown_connections 2023-12-04T05:39:24.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.547+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2998281257 wait complete. 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 Processor -- start 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 -- start start 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3806c960 0x7fbf3814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 0x7fbf380b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf38140200 0x7fbf38158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380b44e0 con 0x7fbf3813cee0 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3813c290 con 0x7fbf3806c960 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.548+0000 7fbf3fb05640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380b3c80 con 0x7fbf38140200 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 0x7fbf380b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 0x7fbf380b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59082/0 (socket says 172.21.15.111:59082) 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 -- 172.21.15.111:0/2319223790 learned_addr learned my addr 172.21.15.111:0/2319223790 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d8a1640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3806c960 0x7fbf3814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3e0a2640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf38140200 0x7fbf38158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 -- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf38140200 msgr2=0x7fbf38158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf38140200 0x7fbf38158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 -- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3806c960 msgr2=0x7fbf3814e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3806c960 0x7fbf3814e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 -- 172.21.15.111:0/2319223790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30009b30 con 0x7fbf3813cee0 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3d0a0640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 0x7fbf380b22b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800d800 tx=0x7fbf2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800b6c0 con 0x7fbf3813cee0 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2800b820 con 0x7fbf3813cee0 2023-12-04T05:39:24.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf380b27f0 con 0x7fbf3813cee0 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800bbb0 con 0x7fbf3813cee0 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.549+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf380b4200 con 0x7fbf3813cee0 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.550+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3813c400 con 0x7fbf3813cee0 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.550+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbf2800bd10 con 0x7fbf3813cee0 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.551+0000 7fbf2e7fc640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf1c06b2a0 0x7fbf1c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.551+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf2808a010 con 0x7fbf3813cee0 2023-12-04T05:39:24.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.551+0000 7fbf3d8a1640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf1c06b2a0 0x7fbf1c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.551+0000 7fbf3d8a1640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf1c06b2a0 0x7fbf1c06d760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf34004f60 tx=0x7fbf34007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.555+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf28054f30 con 0x7fbf3813cee0 2023-12-04T05:39:24.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.720+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf3813fce0 con 0x7fbf1c06b2a0 2023-12-04T05:39:24.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.726+0000 7fbf2e7fc640 1 -- 172.21.15.111:0/2319223790 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264464 (secure 0 0 0) 0x7fbf34003940 con 0x7fbf1c06b2a0 2023-12-04T05:39:24.730 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:24.732 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf1c06b2a0 msgr2=0x7fbf1c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf1c06b2a0 0x7fbf1c06d760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf34004f60 tx=0x7fbf34007e30 comp rx=0 tx=0).stop 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 msgr2=0x7fbf380b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 0x7fbf380b22b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800d800 tx=0x7fbf2800dcc0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 shutdown_connections 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf1c06b2a0 0x7fbf1c06d760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf38140200 0x7fbf38158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3813cee0 0x7fbf380b22b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 --2- 172.21.15.111:0/2319223790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3806c960 0x7fbf3814e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 >> 172.21.15.111:0/2319223790 conn(0x7fbf3806a4c0 msgr2=0x7fbf38150120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 shutdown_connections 2023-12-04T05:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.729+0000 7fbf3fb05640 1 -- 172.21.15.111:0/2319223790 wait complete. 2023-12-04T05:39:24.741 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-04T05:39:24.474192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001592"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519836+0000","last_change":"2023-12-04T05:39:17.207470+0000","last_active":"2023-12-04T05:39:18.519836+0000","last_peered":"2023-12-04T05:39:18.519836+0000","last_clean":"2023-12-04T05:39:18.519836+0000","last_became_active":"2023-12-04T05:39:17.207195+0000","last_became_peered":"2023-12-04T05:39:17.207195+0000","last_unstale":"2023-12-04T05:39:18.519836+0000","last_undegraded":"2023-12-04T05:39:18.519836+0000","last_fullsized":"2023-12-04T05:39:18.519836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:01:28.140633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195840+0000","last_change":"2023-12-04T05:39:17.210099+0000","last_active":"2023-12-04T05:39:18.195840+0000","last_peered":"2023-12-04T05:39:18.195840+0000","last_clean":"2023-12-04T05:39:18.195840+0000","last_became_active":"2023-12-04T05:39:17.209800+0000","last_became_peered":"2023-12-04T05:39:17.209800+0000","last_unstale":"2023-12-04T05:39:18.195840+0000","last_undegraded":"2023-12-04T05:39:18.195840+0000","last_fullsized":"2023-12-04T05:39:18.195840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518434+0000","last_change":"2023-12-04T05:39:17.213084+0000","last_active":"2023-12-04T05:39:18.518434+0000","last_peered":"2023-12-04T05:39:18.518434+0000","last_clean":"2023-12-04T05:39:18.518434+0000","last_became_active":"2023-12-04T05:39:17.212742+0000","last_became_peered":"2023-12-04T05:39:17.212742+0000","last_unstale":"2023-12-04T05:39:18.518434+0000","last_undegraded":"2023-12-04T05:39:18.518434+0000","last_fullsized":"2023-12-04T05:39:18.518434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195184+0000","last_change":"2023-12-04T05:39:17.209715+0000","last_active":"2023-12-04T05:39:18.195184+0000","last_peered":"2023-12-04T05:39:18.195184+0000","last_clean":"2023-12-04T05:39:18.195184+0000","last_became_active":"2023-12-04T05:39:17.209399+0000","last_became_peered":"2023-12-04T05:39:17.209399+0000","last_unstale":"2023-12-04T05:39:18.195184+0000","last_undegraded":"2023-12-04T05:39:18.195184+0000","last_fullsized":"2023-12-04T05:39:18.195184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519183+0000","last_change":"2023-12-04T05:39:17.209067+0000","last_active":"2023-12-04T05:39:18.519183+0000","last_peered":"2023-12-04T05:39:18.519183+0000","last_clean":"2023-12-04T05:39:18.519183+0000","last_became_active":"2023-12-04T05:39:17.208794+0000","last_became_peered":"2023-12-04T05:39:17.208794+0000","last_unstale":"2023-12-04T05:39:18.519183+0000","last_undegraded":"2023-12-04T05:39:18.519183+0000","last_fullsized":"2023-12-04T05:39:18.519183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195035+0000","last_change":"2023-12-04T05:39:17.216846+0000","last_active":"2023-12-04T05:39:18.195035+0000","last_peered":"2023-12-04T05:39:18.195035+0000","last_clean":"2023-12-04T05:39:18.195035+0000","last_became_active":"2023-12-04T05:39:17.216536+0000","last_became_peered":"2023-12-04T05:39:17.216536+0000","last_unstale":"2023-12-04T05:39:18.195035+0000","last_undegraded":"2023-12-04T05:39:18.195035+0000","last_fullsized":"2023-12-04T05:39:18.195035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:53.681504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519109+0000","last_change":"2023-12-04T05:39:17.216228+0000","last_active":"2023-12-04T05:39:18.519109+0000","last_peered":"2023-12-04T05:39:18.519109+0000","last_clean":"2023-12-04T05:39:18.519109+0000","last_became_active":"2023-12-04T05:39:17.215958+0000","last_became_peered":"2023-12-04T05:39:17.215958+0000","last_unstale":"2023-12-04T05:39:18.519109+0000","last_undegraded":"2023-12-04T05:39:18.519109+0000","last_fullsized":"2023-12-04T05:39:18.519109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495336+0000","last_change":"2023-12-04T05:39:17.212872+0000","last_active":"2023-12-04T05:39:18.495336+0000","last_peered":"2023-12-04T05:39:18.495336+0000","last_clean":"2023-12-04T05:39:18.495336+0000","last_became_active":"2023-12-04T05:39:17.212550+0000","last_became_peered":"2023-12-04T05:39:17.212550+0000","last_unstale":"2023-12-04T05:39:18.495336+0000","last_undegraded":"2023-12-04T05:39:18.495336+0000","last_fullsized":"2023-12-04T05:39:18.495336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519658+0000","last_change":"2023-12-04T05:39:17.210290+0000","last_active":"2023-12-04T05:39:18.519658+0000","last_peered":"2023-12-04T05:39:18.519658+0000","last_clean":"2023-12-04T05:39:18.519658+0000","last_became_active":"2023-12-04T05:39:17.210041+0000","last_became_peered":"2023-12-04T05:39:17.210041+0000","last_unstale":"2023-12-04T05:39:18.519658+0000","last_undegraded":"2023-12-04T05:39:18.519658+0000","last_fullsized":"2023-12-04T05:39:18.519658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:01:33.532203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519694+0000","last_change":"2023-12-04T05:39:17.212608+0000","last_active":"2023-12-04T05:39:18.519694+0000","last_peered":"2023-12-04T05:39:18.519694+0000","last_clean":"2023-12-04T05:39:18.519694+0000","last_became_active":"2023-12-04T05:39:17.212249+0000","last_became_peered":"2023-12-04T05:39:17.212249+0000","last_unstale":"2023-12-04T05:39:18.519694+0000","last_undegraded":"2023-12-04T05:39:18.519694+0000","last_fullsized":"2023-12-04T05:39:18.519694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519356+0000","last_change":"2023-12-04T05:39:17.206680+0000","last_active":"2023-12-04T05:39:18.519356+0000","last_peered":"2023-12-04T05:39:18.519356+0000","last_clean":"2023-12-04T05:39:18.519356+0000","last_became_active":"2023-12-04T05:39:17.206326+0000","last_became_peered":"2023-12-04T05:39:17.206326+0000","last_unstale":"2023-12-04T05:39:18.519356+0000","last_undegraded":"2023-12-04T05:39:18.519356+0000","last_fullsized":"2023-12-04T05:39:18.519356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194773+0000","last_change":"2023-12-04T05:39:17.206360+0000","last_active":"2023-12-04T05:39:18.194773+0000","last_peered":"2023-12-04T05:39:18.194773+0000","last_clean":"2023-12-04T05:39:18.194773+0000","last_became_active":"2023-12-04T05:39:17.205989+0000","last_became_peered":"2023-12-04T05:39:17.205989+0000","last_unstale":"2023-12-04T05:39:18.194773+0000","last_undegraded":"2023-12-04T05:39:18.194773+0000","last_fullsized":"2023-12-04T05:39:18.194773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:22:45.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495552+0000","last_change":"2023-12-04T05:39:17.207638+0000","last_active":"2023-12-04T05:39:18.495552+0000","last_peered":"2023-12-04T05:39:18.495552+0000","last_clean":"2023-12-04T05:39:18.495552+0000","last_became_active":"2023-12-04T05:39:17.207368+0000","last_became_peered":"2023-12-04T05:39:17.207368+0000","last_unstale":"2023-12-04T05:39:18.495552+0000","last_undegraded":"2023-12-04T05:39:18.495552+0000","last_fullsized":"2023-12-04T05:39:18.495552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518614+0000","last_change":"2023-12-04T05:39:17.210284+0000","last_active":"2023-12-04T05:39:18.518614+0000","last_peered":"2023-12-04T05:39:18.518614+0000","last_clean":"2023-12-04T05:39:18.518614+0000","last_became_active":"2023-12-04T05:39:17.209989+0000","last_became_peered":"2023-12-04T05:39:17.209989+0000","last_unstale":"2023-12-04T05:39:18.518614+0000","last_undegraded":"2023-12-04T05:39:18.518614+0000","last_fullsized":"2023-12-04T05:39:18.518614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195553+0000","last_change":"2023-12-04T05:39:17.207109+0000","last_active":"2023-12-04T05:39:18.195553+0000","last_peered":"2023-12-04T05:39:18.195553+0000","last_clean":"2023-12-04T05:39:18.195553+0000","last_became_active":"2023-12-04T05:39:17.205722+0000","last_became_peered":"2023-12-04T05:39:17.205722+0000","last_unstale":"2023-12-04T05:39:18.195553+0000","last_undegraded":"2023-12-04T05:39:18.195553+0000","last_fullsized":"2023-12-04T05:39:18.195553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194533+0000","last_change":"2023-12-04T05:39:17.213549+0000","last_active":"2023-12-04T05:39:18.194533+0000","last_peered":"2023-12-04T05:39:18.194533+0000","last_clean":"2023-12-04T05:39:18.194533+0000","last_became_active":"2023-12-04T05:39:17.212758+0000","last_became_peered":"2023-12-04T05:39:17.212758+0000","last_unstale":"2023-12-04T05:39:18.194533+0000","last_undegraded":"2023-12-04T05:39:18.194533+0000","last_fullsized":"2023-12-04T05:39:18.194533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195513+0000","last_change":"2023-12-04T05:39:17.210092+0000","last_active":"2023-12-04T05:39:18.195513+0000","last_peered":"2023-12-04T05:39:18.195513+0000","last_clean":"2023-12-04T05:39:18.195513+0000","last_became_active":"2023-12-04T05:39:17.209828+0000","last_became_peered":"2023-12-04T05:39:17.209828+0000","last_unstale":"2023-12-04T05:39:18.195513+0000","last_undegraded":"2023-12-04T05:39:18.195513+0000","last_fullsized":"2023-12-04T05:39:18.195513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:01:33.532203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518749+0000","last_change":"2023-12-04T05:39:17.206756+0000","last_active":"2023-12-04T05:39:18.518749+0000","last_peered":"2023-12-04T05:39:18.518749+0000","last_clean":"2023-12-04T05:39:18.518749+0000","last_became_active":"2023-12-04T05:39:17.206499+0000","last_became_peered":"2023-12-04T05:39:17.206499+0000","last_unstale":"2023-12-04T05:39:18.518749+0000","last_undegraded":"2023-12-04T05:39:18.518749+0000","last_fullsized":"2023-12-04T05:39:18.518749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:33.381391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519868+0000","last_change":"2023-12-04T05:39:17.213693+0000","last_active":"2023-12-04T05:39:18.519868+0000","last_peered":"2023-12-04T05:39:18.519868+0000","last_clean":"2023-12-04T05:39:18.519868+0000","last_became_active":"2023-12-04T05:39:17.213404+0000","last_became_peered":"2023-12-04T05:39:17.213404+0000","last_unstale":"2023-12-04T05:39:18.519868+0000","last_undegraded":"2023-12-04T05:39:18.519868+0000","last_fullsized":"2023-12-04T05:39:18.519868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:27:18.604272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519451+0000","last_change":"2023-12-04T05:39:17.211735+0000","last_active":"2023-12-04T05:39:18.519451+0000","last_peered":"2023-12-04T05:39:18.519451+0000","last_clean":"2023-12-04T05:39:18.519451+0000","last_became_active":"2023-12-04T05:39:17.211450+0000","last_became_peered":"2023-12-04T05:39:17.211450+0000","last_unstale":"2023-12-04T05:39:18.519451+0000","last_undegraded":"2023-12-04T05:39:18.519451+0000","last_fullsized":"2023-12-04T05:39:18.519451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518422+0000","last_change":"2023-12-04T05:39:17.215901+0000","last_active":"2023-12-04T05:39:18.518422+0000","last_peered":"2023-12-04T05:39:18.518422+0000","last_clean":"2023-12-04T05:39:18.518422+0000","last_became_active":"2023-12-04T05:39:17.215395+0000","last_became_peered":"2023-12-04T05:39:17.215395+0000","last_unstale":"2023-12-04T05:39:18.518422+0000","last_undegraded":"2023-12-04T05:39:18.518422+0000","last_fullsized":"2023-12-04T05:39:18.518422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:01.587525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195031+0000","last_change":"2023-12-04T05:39:17.211134+0000","last_active":"2023-12-04T05:39:18.195031+0000","last_peered":"2023-12-04T05:39:18.195031+0000","last_clean":"2023-12-04T05:39:18.195031+0000","last_became_active":"2023-12-04T05:39:17.210734+0000","last_became_peered":"2023-12-04T05:39:17.210734+0000","last_unstale":"2023-12-04T05:39:18.195031+0000","last_undegraded":"2023-12-04T05:39:18.195031+0000","last_fullsized":"2023-12-04T05:39:18.195031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495425+0000","last_change":"2023-12-04T05:39:17.211310+0000","last_active":"2023-12-04T05:39:18.495425+0000","last_peered":"2023-12-04T05:39:18.495425+0000","last_clean":"2023-12-04T05:39:18.495425+0000","last_became_active":"2023-12-04T05:39:17.210967+0000","last_became_peered":"2023-12-04T05:39:17.210967+0000","last_unstale":"2023-12-04T05:39:18.495425+0000","last_undegraded":"2023-12-04T05:39:18.495425+0000","last_fullsized":"2023-12-04T05:39:18.495425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194980+0000","last_change":"2023-12-04T05:39:17.212493+0000","last_active":"2023-12-04T05:39:18.194980+0000","last_peered":"2023-12-04T05:39:18.194980+0000","last_clean":"2023-12-04T05:39:18.194980+0000","last_became_active":"2023-12-04T05:39:17.212240+0000","last_became_peered":"2023-12-04T05:39:17.212240+0000","last_unstale":"2023-12-04T05:39:18.194980+0000","last_undegraded":"2023-12-04T05:39:18.194980+0000","last_fullsized":"2023-12-04T05:39:18.194980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495743+0000","last_change":"2023-12-04T05:39:17.211088+0000","last_active":"2023-12-04T05:39:18.495743+0000","last_peered":"2023-12-04T05:39:18.495743+0000","last_clean":"2023-12-04T05:39:18.495743+0000","last_became_active":"2023-12-04T05:39:17.210779+0000","last_became_peered":"2023-12-04T05:39:17.210779+0000","last_unstale":"2023-12-04T05:39:18.495743+0000","last_undegraded":"2023-12-04T05:39:18.495743+0000","last_fullsized":"2023-12-04T05:39:18.495743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519676+0000","last_change":"2023-12-04T05:39:17.213404+0000","last_active":"2023-12-04T05:39:18.519676+0000","last_peered":"2023-12-04T05:39:18.519676+0000","last_clean":"2023-12-04T05:39:18.519676+0000","last_became_active":"2023-12-04T05:39:17.213153+0000","last_became_peered":"2023-12-04T05:39:17.213153+0000","last_unstale":"2023-12-04T05:39:18.519676+0000","last_undegraded":"2023-12-04T05:39:18.519676+0000","last_fullsized":"2023-12-04T05:39:18.519676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195771+0000","last_change":"2023-12-04T05:39:17.207997+0000","last_active":"2023-12-04T05:39:18.195771+0000","last_peered":"2023-12-04T05:39:18.195771+0000","last_clean":"2023-12-04T05:39:18.195771+0000","last_became_active":"2023-12-04T05:39:17.207537+0000","last_became_peered":"2023-12-04T05:39:17.207537+0000","last_unstale":"2023-12-04T05:39:18.195771+0000","last_undegraded":"2023-12-04T05:39:18.195771+0000","last_fullsized":"2023-12-04T05:39:18.195771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495833+0000","last_change":"2023-12-04T05:39:17.211215+0000","last_active":"2023-12-04T05:39:18.495833+0000","last_peered":"2023-12-04T05:39:18.495833+0000","last_clean":"2023-12-04T05:39:18.495833+0000","last_became_active":"2023-12-04T05:39:17.210930+0000","last_became_peered":"2023-12-04T05:39:17.210930+0000","last_unstale":"2023-12-04T05:39:18.495833+0000","last_undegraded":"2023-12-04T05:39:18.495833+0000","last_fullsized":"2023-12-04T05:39:18.495833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518602+0000","last_change":"2023-12-04T05:39:17.215388+0000","last_active":"2023-12-04T05:39:18.518602+0000","last_peered":"2023-12-04T05:39:18.518602+0000","last_clean":"2023-12-04T05:39:18.518602+0000","last_became_active":"2023-12-04T05:39:17.215082+0000","last_became_peered":"2023-12-04T05:39:17.215082+0000","last_unstale":"2023-12-04T05:39:18.518602+0000","last_undegraded":"2023-12-04T05:39:18.518602+0000","last_fullsized":"2023-12-04T05:39:18.518602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195233+0000","last_change":"2023-12-04T05:39:17.207162+0000","last_active":"2023-12-04T05:39:18.195233+0000","last_peered":"2023-12-04T05:39:18.195233+0000","last_clean":"2023-12-04T05:39:18.195233+0000","last_became_active":"2023-12-04T05:39:17.206737+0000","last_became_peered":"2023-12-04T05:39:17.206737+0000","last_unstale":"2023-12-04T05:39:18.195233+0000","last_undegraded":"2023-12-04T05:39:18.195233+0000","last_fullsized":"2023-12-04T05:39:18.195233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518154+0000","last_change":"2023-12-04T05:39:17.204908+0000","last_active":"2023-12-04T05:39:18.518154+0000","last_peered":"2023-12-04T05:39:18.518154+0000","last_clean":"2023-12-04T05:39:18.518154+0000","last_became_active":"2023-12-04T05:39:17.204656+0000","last_became_peered":"2023-12-04T05:39:17.204656+0000","last_unstale":"2023-12-04T05:39:18.518154+0000","last_undegraded":"2023-12-04T05:39:18.518154+0000","last_fullsized":"2023-12-04T05:39:18.518154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:33.059240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518987+0000","last_change":"2023-12-04T05:39:17.211113+0000","last_active":"2023-12-04T05:39:18.518987+0000","last_peered":"2023-12-04T05:39:18.518987+0000","last_clean":"2023-12-04T05:39:18.518987+0000","last_became_active":"2023-12-04T05:39:17.210626+0000","last_became_peered":"2023-12-04T05:39:17.210626+0000","last_unstale":"2023-12-04T05:39:18.518987+0000","last_undegraded":"2023-12-04T05:39:18.518987+0000","last_fullsized":"2023-12-04T05:39:18.518987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194380+0000","last_change":"2023-12-04T05:39:17.207261+0000","last_active":"2023-12-04T05:39:18.194380+0000","last_peered":"2023-12-04T05:39:18.194380+0000","last_clean":"2023-12-04T05:39:18.194380+0000","last_became_active":"2023-12-04T05:39:17.206957+0000","last_became_peered":"2023-12-04T05:39:17.206957+0000","last_unstale":"2023-12-04T05:39:18.194380+0000","last_undegraded":"2023-12-04T05:39:18.194380+0000","last_fullsized":"2023-12-04T05:39:18.194380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:38:49.601206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495637+0000","last_change":"2023-12-04T05:39:16.222472+0000","last_active":"2023-12-04T05:39:18.495637+0000","last_peered":"2023-12-04T05:39:18.495637+0000","last_clean":"2023-12-04T05:39:18.495637+0000","last_became_active":"2023-12-04T05:39:16.222136+0000","last_became_peered":"2023-12-04T05:39:16.222136+0000","last_unstale":"2023-12-04T05:39:18.495637+0000","last_undegraded":"2023-12-04T05:39:18.495637+0000","last_fullsized":"2023-12-04T05:39:18.495637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518714+0000","last_change":"2023-12-04T05:39:17.215145+0000","last_active":"2023-12-04T05:39:18.518714+0000","last_peered":"2023-12-04T05:39:18.518714+0000","last_clean":"2023-12-04T05:39:18.518714+0000","last_became_active":"2023-12-04T05:39:17.214877+0000","last_became_peered":"2023-12-04T05:39:17.214877+0000","last_unstale":"2023-12-04T05:39:18.518714+0000","last_undegraded":"2023-12-04T05:39:18.518714+0000","last_fullsized":"2023-12-04T05:39:18.518714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:21.639103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519318+0000","last_change":"2023-12-04T05:39:16.216663+0000","last_active":"2023-12-04T05:39:18.519318+0000","last_peered":"2023-12-04T05:39:18.519318+0000","last_clean":"2023-12-04T05:39:18.519318+0000","last_became_active":"2023-12-04T05:39:16.216408+0000","last_became_peered":"2023-12-04T05:39:16.216408+0000","last_unstale":"2023-12-04T05:39:18.519318+0000","last_undegraded":"2023-12-04T05:39:18.519318+0000","last_fullsized":"2023-12-04T05:39:18.519318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:01:24.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495399+0000","last_change":"2023-12-04T05:39:16.216780+0000","last_active":"2023-12-04T05:39:18.495399+0000","last_peered":"2023-12-04T05:39:18.495399+0000","last_clean":"2023-12-04T05:39:18.495399+0000","last_became_active":"2023-12-04T05:39:16.216486+0000","last_became_peered":"2023-12-04T05:39:16.216486+0000","last_unstale":"2023-12-04T05:39:18.495399+0000","last_undegraded":"2023-12-04T05:39:18.495399+0000","last_fullsized":"2023-12-04T05:39:18.495399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:00.582466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495379+0000","last_change":"2023-12-04T05:39:17.215850+0000","last_active":"2023-12-04T05:39:18.495379+0000","last_peered":"2023-12-04T05:39:18.495379+0000","last_clean":"2023-12-04T05:39:18.495379+0000","last_became_active":"2023-12-04T05:39:17.215557+0000","last_became_peered":"2023-12-04T05:39:17.215557+0000","last_unstale":"2023-12-04T05:39:18.495379+0000","last_undegraded":"2023-12-04T05:39:18.495379+0000","last_fullsized":"2023-12-04T05:39:18.495379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194854+0000","last_change":"2023-12-04T05:39:16.213564+0000","last_active":"2023-12-04T05:39:18.194854+0000","last_peered":"2023-12-04T05:39:18.194854+0000","last_clean":"2023-12-04T05:39:18.194854+0000","last_became_active":"2023-12-04T05:39:16.213288+0000","last_became_peered":"2023-12-04T05:39:16.213288+0000","last_unstale":"2023-12-04T05:39:18.194854+0000","last_undegraded":"2023-12-04T05:39:18.194854+0000","last_fullsized":"2023-12-04T05:39:18.194854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195514+0000","last_change":"2023-12-04T05:39:17.208143+0000","last_active":"2023-12-04T05:39:18.195514+0000","last_peered":"2023-12-04T05:39:18.195514+0000","last_clean":"2023-12-04T05:39:18.195514+0000","last_became_active":"2023-12-04T05:39:17.207883+0000","last_became_peered":"2023-12-04T05:39:17.207883+0000","last_unstale":"2023-12-04T05:39:18.195514+0000","last_undegraded":"2023-12-04T05:39:18.195514+0000","last_fullsized":"2023-12-04T05:39:18.195514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:06.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519244+0000","last_change":"2023-12-04T05:39:16.217144+0000","last_active":"2023-12-04T05:39:18.519244+0000","last_peered":"2023-12-04T05:39:18.519244+0000","last_clean":"2023-12-04T05:39:18.519244+0000","last_became_active":"2023-12-04T05:39:16.216759+0000","last_became_peered":"2023-12-04T05:39:16.216759+0000","last_unstale":"2023-12-04T05:39:18.519244+0000","last_undegraded":"2023-12-04T05:39:18.519244+0000","last_fullsized":"2023-12-04T05:39:18.519244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:22:44.119471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195253+0000","last_change":"2023-12-04T05:39:17.211977+0000","last_active":"2023-12-04T05:39:18.195253+0000","last_peered":"2023-12-04T05:39:18.195253+0000","last_clean":"2023-12-04T05:39:18.195253+0000","last_became_active":"2023-12-04T05:39:17.211707+0000","last_became_peered":"2023-12-04T05:39:17.211707+0000","last_unstale":"2023-12-04T05:39:18.195253+0000","last_undegraded":"2023-12-04T05:39:18.195253+0000","last_fullsized":"2023-12-04T05:39:18.195253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195229+0000","last_change":"2023-12-04T05:39:16.212865+0000","last_active":"2023-12-04T05:39:18.195229+0000","last_peered":"2023-12-04T05:39:18.195229+0000","last_clean":"2023-12-04T05:39:18.195229+0000","last_became_active":"2023-12-04T05:39:16.212574+0000","last_became_peered":"2023-12-04T05:39:16.212574+0000","last_unstale":"2023-12-04T05:39:18.195229+0000","last_undegraded":"2023-12-04T05:39:18.195229+0000","last_fullsized":"2023-12-04T05:39:18.195229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:17.975393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518554+0000","last_change":"2023-12-04T05:39:17.210657+0000","last_active":"2023-12-04T05:39:18.518554+0000","last_peered":"2023-12-04T05:39:18.518554+0000","last_clean":"2023-12-04T05:39:18.518554+0000","last_became_active":"2023-12-04T05:39:17.210197+0000","last_became_peered":"2023-12-04T05:39:17.210197+0000","last_unstale":"2023-12-04T05:39:18.518554+0000","last_undegraded":"2023-12-04T05:39:18.518554+0000","last_fullsized":"2023-12-04T05:39:18.518554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195730+0000","last_change":"2023-12-04T05:39:16.219612+0000","last_active":"2023-12-04T05:39:18.195730+0000","last_peered":"2023-12-04T05:39:18.195730+0000","last_clean":"2023-12-04T05:39:18.195730+0000","last_became_active":"2023-12-04T05:39:16.219305+0000","last_became_peered":"2023-12-04T05:39:16.219305+0000","last_unstale":"2023-12-04T05:39:18.195730+0000","last_undegraded":"2023-12-04T05:39:18.195730+0000","last_fullsized":"2023-12-04T05:39:18.195730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194796+0000","last_change":"2023-12-04T05:39:17.202594+0000","last_active":"2023-12-04T05:39:18.194796+0000","last_peered":"2023-12-04T05:39:18.194796+0000","last_clean":"2023-12-04T05:39:18.194796+0000","last_became_active":"2023-12-04T05:39:17.202079+0000","last_became_peered":"2023-12-04T05:39:17.202079+0000","last_unstale":"2023-12-04T05:39:18.194796+0000","last_undegraded":"2023-12-04T05:39:18.194796+0000","last_fullsized":"2023-12-04T05:39:18.194796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:33.059240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519559+0000","last_change":"2023-12-04T05:39:16.212413+0000","last_active":"2023-12-04T05:39:18.519559+0000","last_peered":"2023-12-04T05:39:18.519559+0000","last_clean":"2023-12-04T05:39:18.519559+0000","last_became_active":"2023-12-04T05:39:16.212127+0000","last_became_peered":"2023-12-04T05:39:16.212127+0000","last_unstale":"2023-12-04T05:39:18.519559+0000","last_undegraded":"2023-12-04T05:39:18.519559+0000","last_fullsized":"2023-12-04T05:39:18.519559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519613+0000","last_change":"2023-12-04T05:39:17.208325+0000","last_active":"2023-12-04T05:39:18.519613+0000","last_peered":"2023-12-04T05:39:18.519613+0000","last_clean":"2023-12-04T05:39:18.519613+0000","last_became_active":"2023-12-04T05:39:17.208060+0000","last_became_peered":"2023-12-04T05:39:17.208060+0000","last_unstale":"2023-12-04T05:39:18.519613+0000","last_undegraded":"2023-12-04T05:39:18.519613+0000","last_fullsized":"2023-12-04T05:39:18.519613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:56:49.040036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518855+0000","last_change":"2023-12-04T05:39:16.217018+0000","last_active":"2023-12-04T05:39:18.518855+0000","last_peered":"2023-12-04T05:39:18.518855+0000","last_clean":"2023-12-04T05:39:18.518855+0000","last_became_active":"2023-12-04T05:39:16.216772+0000","last_became_peered":"2023-12-04T05:39:16.216772+0000","last_unstale":"2023-12-04T05:39:18.518855+0000","last_undegraded":"2023-12-04T05:39:18.518855+0000","last_fullsized":"2023-12-04T05:39:18.518855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:32.054181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518705+0000","last_change":"2023-12-04T05:39:17.213778+0000","last_active":"2023-12-04T05:39:18.518705+0000","last_peered":"2023-12-04T05:39:18.518705+0000","last_clean":"2023-12-04T05:39:18.518705+0000","last_became_active":"2023-12-04T05:39:17.213520+0000","last_became_peered":"2023-12-04T05:39:17.213520+0000","last_unstale":"2023-12-04T05:39:18.518705+0000","last_undegraded":"2023-12-04T05:39:18.518705+0000","last_fullsized":"2023-12-04T05:39:18.518705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195291+0000","last_change":"2023-12-04T05:39:16.212009+0000","last_active":"2023-12-04T05:39:18.195291+0000","last_peered":"2023-12-04T05:39:18.195291+0000","last_clean":"2023-12-04T05:39:18.195291+0000","last_became_active":"2023-12-04T05:39:16.211611+0000","last_became_peered":"2023-12-04T05:39:16.211611+0000","last_unstale":"2023-12-04T05:39:18.195291+0000","last_undegraded":"2023-12-04T05:39:18.195291+0000","last_fullsized":"2023-12-04T05:39:18.195291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:24.393125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519548+0000","last_change":"2023-12-04T05:39:17.210008+0000","last_active":"2023-12-04T05:39:18.519548+0000","last_peered":"2023-12-04T05:39:18.519548+0000","last_clean":"2023-12-04T05:39:18.519548+0000","last_became_active":"2023-12-04T05:39:17.209767+0000","last_became_peered":"2023-12-04T05:39:17.209767+0000","last_unstale":"2023-12-04T05:39:18.519548+0000","last_undegraded":"2023-12-04T05:39:18.519548+0000","last_fullsized":"2023-12-04T05:39:18.519548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:35:27.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519053+0000","last_change":"2023-12-04T05:39:16.219504+0000","last_active":"2023-12-04T05:39:18.519053+0000","last_peered":"2023-12-04T05:39:18.519053+0000","last_clean":"2023-12-04T05:39:18.519053+0000","last_became_active":"2023-12-04T05:39:16.219241+0000","last_became_peered":"2023-12-04T05:39:16.219241+0000","last_unstale":"2023-12-04T05:39:18.519053+0000","last_undegraded":"2023-12-04T05:39:18.519053+0000","last_fullsized":"2023-12-04T05:39:18.519053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.913440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519043+0000","last_change":"2023-12-04T05:39:17.210165+0000","last_active":"2023-12-04T05:39:18.519043+0000","last_peered":"2023-12-04T05:39:18.519043+0000","last_clean":"2023-12-04T05:39:18.519043+0000","last_became_active":"2023-12-04T05:39:17.209917+0000","last_became_peered":"2023-12-04T05:39:17.209917+0000","last_unstale":"2023-12-04T05:39:18.519043+0000","last_undegraded":"2023-12-04T05:39:18.519043+0000","last_fullsized":"2023-12-04T05:39:18.519043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518077+0000","last_change":"2023-12-04T05:39:16.216340+0000","last_active":"2023-12-04T05:39:18.518077+0000","last_peered":"2023-12-04T05:39:18.518077+0000","last_clean":"2023-12-04T05:39:18.518077+0000","last_became_active":"2023-12-04T05:39:16.216034+0000","last_became_peered":"2023-12-04T05:39:16.216034+0000","last_unstale":"2023-12-04T05:39:18.518077+0000","last_undegraded":"2023-12-04T05:39:18.518077+0000","last_fullsized":"2023-12-04T05:39:18.518077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.913440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519028+0000","last_change":"2023-12-04T05:39:17.216087+0000","last_active":"2023-12-04T05:39:18.519028+0000","last_peered":"2023-12-04T05:39:18.519028+0000","last_clean":"2023-12-04T05:39:18.519028+0000","last_became_active":"2023-12-04T05:39:17.215420+0000","last_became_peered":"2023-12-04T05:39:17.215420+0000","last_unstale":"2023-12-04T05:39:18.519028+0000","last_undegraded":"2023-12-04T05:39:18.519028+0000","last_fullsized":"2023-12-04T05:39:18.519028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:13:47.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194623+0000","last_change":"2023-12-04T05:39:16.218872+0000","last_active":"2023-12-04T05:39:18.194623+0000","last_peered":"2023-12-04T05:39:18.194623+0000","last_clean":"2023-12-04T05:39:18.194623+0000","last_became_active":"2023-12-04T05:39:16.218609+0000","last_became_peered":"2023-12-04T05:39:16.218609+0000","last_unstale":"2023-12-04T05:39:18.194623+0000","last_undegraded":"2023-12-04T05:39:18.194623+0000","last_fullsized":"2023-12-04T05:39:18.194623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T05:42:48.541211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519439+0000","last_change":"2023-12-04T05:39:17.209465+0000","last_active":"2023-12-04T05:39:18.519439+0000","last_peered":"2023-12-04T05:39:18.519439+0000","last_clean":"2023-12-04T05:39:18.519439+0000","last_became_active":"2023-12-04T05:39:17.209210+0000","last_became_peered":"2023-12-04T05:39:17.209210+0000","last_unstale":"2023-12-04T05:39:18.519439+0000","last_undegraded":"2023-12-04T05:39:18.519439+0000","last_fullsized":"2023-12-04T05:39:18.519439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:58:00.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518796+0000","last_change":"2023-12-04T05:39:16.212731+0000","last_active":"2023-12-04T05:39:18.518796+0000","last_peered":"2023-12-04T05:39:18.518796+0000","last_clean":"2023-12-04T05:39:18.518796+0000","last_became_active":"2023-12-04T05:39:16.212398+0000","last_became_peered":"2023-12-04T05:39:16.212398+0000","last_unstale":"2023-12-04T05:39:18.518796+0000","last_undegraded":"2023-12-04T05:39:18.518796+0000","last_fullsized":"2023-12-04T05:39:18.518796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:13.519031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519792+0000","last_change":"2023-12-04T05:39:17.215878+0000","last_active":"2023-12-04T05:39:18.519792+0000","last_peered":"2023-12-04T05:39:18.519792+0000","last_clean":"2023-12-04T05:39:18.519792+0000","last_became_active":"2023-12-04T05:39:17.215600+0000","last_became_peered":"2023-12-04T05:39:17.215600+0000","last_unstale":"2023-12-04T05:39:18.519792+0000","last_undegraded":"2023-12-04T05:39:18.519792+0000","last_fullsized":"2023-12-04T05:39:18.519792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:21:50.865362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195801+0000","last_change":"2023-12-04T05:39:16.219401+0000","last_active":"2023-12-04T05:39:18.195801+0000","last_peered":"2023-12-04T05:39:18.195801+0000","last_clean":"2023-12-04T05:39:18.195801+0000","last_became_active":"2023-12-04T05:39:16.219096+0000","last_became_peered":"2023-12-04T05:39:16.219096+0000","last_unstale":"2023-12-04T05:39:18.195801+0000","last_undegraded":"2023-12-04T05:39:18.195801+0000","last_fullsized":"2023-12-04T05:39:18.195801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195394+0000","last_change":"2023-12-04T05:39:17.208978+0000","last_active":"2023-12-04T05:39:18.195394+0000","last_peered":"2023-12-04T05:39:18.195394+0000","last_clean":"2023-12-04T05:39:18.195394+0000","last_became_active":"2023-12-04T05:39:17.208163+0000","last_became_peered":"2023-12-04T05:39:17.208163+0000","last_unstale":"2023-12-04T05:39:18.195394+0000","last_undegraded":"2023-12-04T05:39:18.195394+0000","last_fullsized":"2023-12-04T05:39:18.195394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:35:27.121477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518395+0000","last_change":"2023-12-04T05:39:16.216610+0000","last_active":"2023-12-04T05:39:18.518395+0000","last_peered":"2023-12-04T05:39:18.518395+0000","last_clean":"2023-12-04T05:39:18.518395+0000","last_became_active":"2023-12-04T05:39:16.216343+0000","last_became_peered":"2023-12-04T05:39:16.216343+0000","last_unstale":"2023-12-04T05:39:18.518395+0000","last_undegraded":"2023-12-04T05:39:18.518395+0000","last_fullsized":"2023-12-04T05:39:18.518395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:06:28.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519731+0000","last_change":"2023-12-04T05:39:17.206092+0000","last_active":"2023-12-04T05:39:18.519731+0000","last_peered":"2023-12-04T05:39:18.519731+0000","last_clean":"2023-12-04T05:39:18.519731+0000","last_became_active":"2023-12-04T05:39:17.205430+0000","last_became_peered":"2023-12-04T05:39:17.205430+0000","last_unstale":"2023-12-04T05:39:18.519731+0000","last_undegraded":"2023-12-04T05:39:18.519731+0000","last_fullsized":"2023-12-04T05:39:18.519731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:28:25.883452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-04T05:39:17.200250+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:39:17.200250+0000","last_peered":"2023-12-04T05:39:17.200250+0000","last_clean":"2023-12-04T05:39:17.200250+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:39:17.200250+0000","last_undegraded":"2023-12-04T05:39:17.200250+0000","last_fullsized":"2023-12-04T05:39:17.200250+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194896+0000","last_change":"2023-12-04T05:39:16.211697+0000","last_active":"2023-12-04T05:39:18.194896+0000","last_peered":"2023-12-04T05:39:18.194896+0000","last_clean":"2023-12-04T05:39:18.194896+0000","last_became_active":"2023-12-04T05:39:16.211421+0000","last_became_peered":"2023-12-04T05:39:16.211421+0000","last_unstale":"2023-12-04T05:39:18.194896+0000","last_undegraded":"2023-12-04T05:39:18.194896+0000","last_fullsized":"2023-12-04T05:39:18.194896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:33:05.599778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495509+0000","last_change":"2023-12-04T05:39:17.212309+0000","last_active":"2023-12-04T05:39:18.495509+0000","last_peered":"2023-12-04T05:39:18.495509+0000","last_clean":"2023-12-04T05:39:18.495509+0000","last_became_active":"2023-12-04T05:39:17.212069+0000","last_became_peered":"2023-12-04T05:39:17.212069+0000","last_unstale":"2023-12-04T05:39:18.495509+0000","last_undegraded":"2023-12-04T05:39:18.495509+0000","last_fullsized":"2023-12-04T05:39:18.495509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:23:05.631504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518862+0000","last_change":"2023-12-04T05:39:16.222456+0000","last_active":"2023-12-04T05:39:18.518862+0000","last_peered":"2023-12-04T05:39:18.518862+0000","last_clean":"2023-12-04T05:39:18.518862+0000","last_became_active":"2023-12-04T05:39:16.222007+0000","last_became_peered":"2023-12-04T05:39:16.222007+0000","last_unstale":"2023-12-04T05:39:18.518862+0000","last_undegraded":"2023-12-04T05:39:18.518862+0000","last_fullsized":"2023-12-04T05:39:18.518862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518357+0000","last_change":"2023-12-04T05:39:17.209347+0000","last_active":"2023-12-04T05:39:18.518357+0000","last_peered":"2023-12-04T05:39:18.518357+0000","last_clean":"2023-12-04T05:39:18.518357+0000","last_became_active":"2023-12-04T05:39:17.209044+0000","last_became_peered":"2023-12-04T05:39:17.209044+0000","last_unstale":"2023-12-04T05:39:18.518357+0000","last_undegraded":"2023-12-04T05:39:18.518357+0000","last_fullsized":"2023-12-04T05:39:18.518357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:40.539141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518478+0000","last_change":"2023-12-04T05:39:16.213448+0000","last_active":"2023-12-04T05:39:18.518478+0000","last_peered":"2023-12-04T05:39:18.518478+0000","last_clean":"2023-12-04T05:39:18.518478+0000","last_became_active":"2023-12-04T05:39:16.212426+0000","last_became_peered":"2023-12-04T05:39:16.212426+0000","last_unstale":"2023-12-04T05:39:18.518478+0000","last_undegraded":"2023-12-04T05:39:18.518478+0000","last_fullsized":"2023-12-04T05:39:18.518478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:00:49.392004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195538+0000","last_change":"2023-12-04T05:39:17.206807+0000","last_active":"2023-12-04T05:39:18.195538+0000","last_peered":"2023-12-04T05:39:18.195538+0000","last_clean":"2023-12-04T05:39:18.195538+0000","last_became_active":"2023-12-04T05:39:17.206502+0000","last_became_peered":"2023-12-04T05:39:17.206502+0000","last_unstale":"2023-12-04T05:39:18.195538+0000","last_undegraded":"2023-12-04T05:39:18.195538+0000","last_fullsized":"2023-12-04T05:39:18.195538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195374+0000","last_change":"2023-12-04T05:39:16.216126+0000","last_active":"2023-12-04T05:39:18.195374+0000","last_peered":"2023-12-04T05:39:18.195374+0000","last_clean":"2023-12-04T05:39:18.195374+0000","last_became_active":"2023-12-04T05:39:16.215729+0000","last_became_peered":"2023-12-04T05:39:16.215729+0000","last_unstale":"2023-12-04T05:39:18.195374+0000","last_undegraded":"2023-12-04T05:39:18.195374+0000","last_fullsized":"2023-12-04T05:39:18.195374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:18.392982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194711+0000","last_change":"2023-12-04T05:39:17.215887+0000","last_active":"2023-12-04T05:39:18.194711+0000","last_peered":"2023-12-04T05:39:18.194711+0000","last_clean":"2023-12-04T05:39:18.194711+0000","last_became_active":"2023-12-04T05:39:17.215638+0000","last_became_peered":"2023-12-04T05:39:17.215638+0000","last_unstale":"2023-12-04T05:39:18.194711+0000","last_undegraded":"2023-12-04T05:39:18.194711+0000","last_fullsized":"2023-12-04T05:39:18.194711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:01:25.259427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195699+0000","last_change":"2023-12-04T05:39:16.222396+0000","last_active":"2023-12-04T05:39:18.195699+0000","last_peered":"2023-12-04T05:39:18.195699+0000","last_clean":"2023-12-04T05:39:18.195699+0000","last_became_active":"2023-12-04T05:39:16.222118+0000","last_became_peered":"2023-12-04T05:39:16.222118+0000","last_unstale":"2023-12-04T05:39:18.195699+0000","last_undegraded":"2023-12-04T05:39:18.195699+0000","last_fullsized":"2023-12-04T05:39:18.195699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195595+0000","last_change":"2023-12-04T05:39:17.211083+0000","last_active":"2023-12-04T05:39:18.195595+0000","last_peered":"2023-12-04T05:39:18.195595+0000","last_clean":"2023-12-04T05:39:18.195595+0000","last_became_active":"2023-12-04T05:39:17.210751+0000","last_became_peered":"2023-12-04T05:39:17.210751+0000","last_unstale":"2023-12-04T05:39:18.195595+0000","last_undegraded":"2023-12-04T05:39:18.195595+0000","last_fullsized":"2023-12-04T05:39:18.195595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:28:25.883452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519823+0000","last_change":"2023-12-04T05:39:16.222777+0000","last_active":"2023-12-04T05:39:18.519823+0000","last_peered":"2023-12-04T05:39:18.519823+0000","last_clean":"2023-12-04T05:39:18.519823+0000","last_became_active":"2023-12-04T05:39:16.222487+0000","last_became_peered":"2023-12-04T05:39:16.222487+0000","last_unstale":"2023-12-04T05:39:18.519823+0000","last_undegraded":"2023-12-04T05:39:18.519823+0000","last_fullsized":"2023-12-04T05:39:18.519823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:56:20.634044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195752+0000","last_change":"2023-12-04T05:39:17.206784+0000","last_active":"2023-12-04T05:39:18.195752+0000","last_peered":"2023-12-04T05:39:18.195752+0000","last_clean":"2023-12-04T05:39:18.195752+0000","last_became_active":"2023-12-04T05:39:17.206468+0000","last_became_peered":"2023-12-04T05:39:17.206468+0000","last_unstale":"2023-12-04T05:39:18.195752+0000","last_undegraded":"2023-12-04T05:39:18.195752+0000","last_fullsized":"2023-12-04T05:39:18.195752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:45:19.398041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:19.490804+0000","last_change":"2023-12-04T05:39:16.219828+0000","last_active":"2023-12-04T05:39:19.490804+0000","last_peered":"2023-12-04T05:39:19.490804+0000","last_clean":"2023-12-04T05:39:19.490804+0000","last_became_active":"2023-12-04T05:39:16.219564+0000","last_became_peered":"2023-12-04T05:39:16.219564+0000","last_unstale":"2023-12-04T05:39:19.490804+0000","last_undegraded":"2023-12-04T05:39:19.490804+0000","last_fullsized":"2023-12-04T05:39:19.490804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:32:00.582466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495806+0000","last_change":"2023-12-04T05:39:17.213462+0000","last_active":"2023-12-04T05:39:18.495806+0000","last_peered":"2023-12-04T05:39:18.495806+0000","last_clean":"2023-12-04T05:39:18.495806+0000","last_became_active":"2023-12-04T05:39:17.213163+0000","last_became_peered":"2023-12-04T05:39:17.213163+0000","last_unstale":"2023-12-04T05:39:18.495806+0000","last_undegraded":"2023-12-04T05:39:18.495806+0000","last_fullsized":"2023-12-04T05:39:18.495806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:05:26.680984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519346+0000","last_change":"2023-12-04T05:39:16.225197+0000","last_active":"2023-12-04T05:39:18.519346+0000","last_peered":"2023-12-04T05:39:18.519346+0000","last_clean":"2023-12-04T05:39:18.519346+0000","last_became_active":"2023-12-04T05:39:16.224896+0000","last_became_peered":"2023-12-04T05:39:16.224896+0000","last_unstale":"2023-12-04T05:39:18.519346+0000","last_undegraded":"2023-12-04T05:39:18.519346+0000","last_fullsized":"2023-12-04T05:39:18.519346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518458+0000","last_change":"2023-12-04T05:39:17.211876+0000","last_active":"2023-12-04T05:39:18.518458+0000","last_peered":"2023-12-04T05:39:18.518458+0000","last_clean":"2023-12-04T05:39:18.518458+0000","last_became_active":"2023-12-04T05:39:17.211580+0000","last_became_peered":"2023-12-04T05:39:17.211580+0000","last_unstale":"2023-12-04T05:39:18.518458+0000","last_undegraded":"2023-12-04T05:39:18.518458+0000","last_fullsized":"2023-12-04T05:39:18.518458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:18.980452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518857+0000","last_change":"2023-12-04T05:39:16.222816+0000","last_active":"2023-12-04T05:39:18.518857+0000","last_peered":"2023-12-04T05:39:18.518857+0000","last_clean":"2023-12-04T05:39:18.518857+0000","last_became_active":"2023-12-04T05:39:16.222453+0000","last_became_peered":"2023-12-04T05:39:16.222453+0000","last_unstale":"2023-12-04T05:39:18.518857+0000","last_undegraded":"2023-12-04T05:39:18.518857+0000","last_fullsized":"2023-12-04T05:39:18.518857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:02:39.534081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.194429+0000","last_change":"2023-12-04T05:39:17.204065+0000","last_active":"2023-12-04T05:39:18.194429+0000","last_peered":"2023-12-04T05:39:18.194429+0000","last_clean":"2023-12-04T05:39:18.194429+0000","last_became_active":"2023-12-04T05:39:17.203770+0000","last_became_peered":"2023-12-04T05:39:17.203770+0000","last_unstale":"2023-12-04T05:39:18.194429+0000","last_undegraded":"2023-12-04T05:39:18.194429+0000","last_fullsized":"2023-12-04T05:39:18.194429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:00:50.397064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195575+0000","last_change":"2023-12-04T05:39:16.223148+0000","last_active":"2023-12-04T05:39:18.195575+0000","last_peered":"2023-12-04T05:39:18.195575+0000","last_clean":"2023-12-04T05:39:18.195575+0000","last_became_active":"2023-12-04T05:39:16.222884+0000","last_became_peered":"2023-12-04T05:39:16.222884+0000","last_unstale":"2023-12-04T05:39:18.195575+0000","last_undegraded":"2023-12-04T05:39:18.195575+0000","last_fullsized":"2023-12-04T05:39:18.195575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:48:32.054181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495446+0000","last_change":"2023-12-04T05:39:17.207953+0000","last_active":"2023-12-04T05:39:18.495446+0000","last_peered":"2023-12-04T05:39:18.495446+0000","last_clean":"2023-12-04T05:39:18.495446+0000","last_became_active":"2023-12-04T05:39:17.207612+0000","last_became_peered":"2023-12-04T05:39:17.207612+0000","last_unstale":"2023-12-04T05:39:18.495446+0000","last_undegraded":"2023-12-04T05:39:18.495446+0000","last_fullsized":"2023-12-04T05:39:18.495446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:06.402596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195660+0000","last_change":"2023-12-04T05:39:16.216886+0000","last_active":"2023-12-04T05:39:18.195660+0000","last_peered":"2023-12-04T05:39:18.195660+0000","last_clean":"2023-12-04T05:39:18.195660+0000","last_became_active":"2023-12-04T05:39:16.215666+0000","last_became_peered":"2023-12-04T05:39:16.215666+0000","last_unstale":"2023-12-04T05:39:18.195660+0000","last_undegraded":"2023-12-04T05:39:18.195660+0000","last_fullsized":"2023-12-04T05:39:18.195660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495633+0000","last_change":"2023-12-04T05:39:17.216210+0000","last_active":"2023-12-04T05:39:18.495633+0000","last_peered":"2023-12-04T05:39:18.495633+0000","last_clean":"2023-12-04T05:39:18.495633+0000","last_became_active":"2023-12-04T05:39:17.215926+0000","last_became_peered":"2023-12-04T05:39:17.215926+0000","last_unstale":"2023-12-04T05:39:18.495633+0000","last_undegraded":"2023-12-04T05:39:18.495633+0000","last_fullsized":"2023-12-04T05:39:18.495633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:53.681504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519432+0000","last_change":"2023-12-04T05:39:16.222353+0000","last_active":"2023-12-04T05:39:18.519432+0000","last_peered":"2023-12-04T05:39:18.519432+0000","last_clean":"2023-12-04T05:39:18.519432+0000","last_became_active":"2023-12-04T05:39:16.221956+0000","last_became_peered":"2023-12-04T05:39:16.221956+0000","last_unstale":"2023-12-04T05:39:18.519432+0000","last_undegraded":"2023-12-04T05:39:18.519432+0000","last_fullsized":"2023-12-04T05:39:18.519432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:38:59.579855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519216+0000","last_change":"2023-12-04T05:39:17.210884+0000","last_active":"2023-12-04T05:39:18.519216+0000","last_peered":"2023-12-04T05:39:18.519216+0000","last_clean":"2023-12-04T05:39:18.519216+0000","last_became_active":"2023-12-04T05:39:17.210601+0000","last_became_peered":"2023-12-04T05:39:17.210601+0000","last_unstale":"2023-12-04T05:39:18.519216+0000","last_undegraded":"2023-12-04T05:39:18.519216+0000","last_fullsized":"2023-12-04T05:39:18.519216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:25.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195487+0000","last_change":"2023-12-04T05:39:17.204261+0000","last_active":"2023-12-04T05:39:18.195487+0000","last_peered":"2023-12-04T05:39:18.195487+0000","last_clean":"2023-12-04T05:39:18.195487+0000","last_became_active":"2023-12-04T05:39:17.203985+0000","last_became_peered":"2023-12-04T05:39:17.203985+0000","last_unstale":"2023-12-04T05:39:18.195487+0000","last_undegraded":"2023-12-04T05:39:18.195487+0000","last_fullsized":"2023-12-04T05:39:18.195487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:56:49.040036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519438+0000","last_change":"2023-12-04T05:39:16.222742+0000","last_active":"2023-12-04T05:39:18.519438+0000","last_peered":"2023-12-04T05:39:18.519438+0000","last_clean":"2023-12-04T05:39:18.519438+0000","last_became_active":"2023-12-04T05:39:16.222073+0000","last_became_peered":"2023-12-04T05:39:16.222073+0000","last_unstale":"2023-12-04T05:39:18.519438+0000","last_undegraded":"2023-12-04T05:39:18.519438+0000","last_fullsized":"2023-12-04T05:39:18.519438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.195197+0000","last_change":"2023-12-04T05:39:17.204727+0000","last_active":"2023-12-04T05:39:18.195197+0000","last_peered":"2023-12-04T05:39:18.195197+0000","last_clean":"2023-12-04T05:39:18.195197+0000","last_became_active":"2023-12-04T05:39:17.203997+0000","last_became_peered":"2023-12-04T05:39:17.203997+0000","last_unstale":"2023-12-04T05:39:18.195197+0000","last_undegraded":"2023-12-04T05:39:18.195197+0000","last_fullsized":"2023-12-04T05:39:18.195197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.519715+0000","last_change":"2023-12-04T05:39:16.223432+0000","last_active":"2023-12-04T05:39:18.519715+0000","last_peered":"2023-12-04T05:39:18.519715+0000","last_clean":"2023-12-04T05:39:18.519715+0000","last_became_active":"2023-12-04T05:39:16.222966+0000","last_became_peered":"2023-12-04T05:39:16.222966+0000","last_unstale":"2023-12-04T05:39:18.519715+0000","last_undegraded":"2023-12-04T05:39:18.519715+0000","last_fullsized":"2023-12-04T05:39:18.519715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:57:24.393125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518460+0000","last_change":"2023-12-04T05:39:17.214833+0000","last_active":"2023-12-04T05:39:18.518460+0000","last_peered":"2023-12-04T05:39:18.518460+0000","last_clean":"2023-12-04T05:39:18.518460+0000","last_became_active":"2023-12-04T05:39:17.214570+0000","last_became_peered":"2023-12-04T05:39:17.214570+0000","last_unstale":"2023-12-04T05:39:18.518460+0000","last_undegraded":"2023-12-04T05:39:18.518460+0000","last_fullsized":"2023-12-04T05:39:18.518460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:24.110066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495773+0000","last_change":"2023-12-04T05:39:16.218785+0000","last_active":"2023-12-04T05:39:18.495773+0000","last_peered":"2023-12-04T05:39:18.495773+0000","last_clean":"2023-12-04T05:39:18.495773+0000","last_became_active":"2023-12-04T05:39:16.218480+0000","last_became_peered":"2023-12-04T05:39:16.218480+0000","last_unstale":"2023-12-04T05:39:18.495773+0000","last_undegraded":"2023-12-04T05:39:18.495773+0000","last_fullsized":"2023-12-04T05:39:18.495773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:10:23.105006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.495766+0000","last_change":"2023-12-04T05:39:17.208223+0000","last_active":"2023-12-04T05:39:18.495766+0000","last_peered":"2023-12-04T05:39:18.495766+0000","last_clean":"2023-12-04T05:39:18.495766+0000","last_became_active":"2023-12-04T05:39:17.207977+0000","last_became_peered":"2023-12-04T05:39:17.207977+0000","last_unstale":"2023-12-04T05:39:18.495766+0000","last_undegraded":"2023-12-04T05:39:18.495766+0000","last_fullsized":"2023-12-04T05:39:18.495766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:16.197699+0000","last_clean_scrub_stamp":"2023-12-04T05:39:16.197699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:23:56.839683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-04T05:39:18.518609+0000","last_change":"2023-12-04T05:39:16.222709+0000","last_active":"2023-12-04T05:39:18.518609+0000","last_peered":"2023-12-04T05:39:18.518609+0000","last_clean":"2023-12-04T05:39:18.518609+0000","last_became_active":"2023-12-04T05:39:16.222155+0000","last_became_peered":"2023-12-04T05:39:16.222155+0000","last_unstale":"2023-12-04T05:39:18.518609+0000","last_undegraded":"2023-12-04T05:39:18.518609+0000","last_fullsized":"2023-12-04T05:39:18.518609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:15.192640+0000","last_clean_scrub_stamp":"2023-12-04T05:39:15.192640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:59:32.376332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.613}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.04}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.193}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.219}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:39:24.746 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-04T05:39:24.746 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-04T05:39:24.746 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-04T05:39:24.746 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-04T05:39:24.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.879+0000 7fcfddfa9640 1 Processor -- start 2023-12-04T05:39:24.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.879+0000 7fcfddfa9640 1 -- start start 2023-12-04T05:39:24.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfddfa9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfd806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfddfa9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80cadf0 con 0x7fcfd8146a50 2023-12-04T05:39:24.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfddfa9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80ce7c0 con 0x7fcfd8161a80 2023-12-04T05:39:24.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfddfa9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8161f10 con 0x7fcfd80b32b0 2023-12-04T05:39:24.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfd77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcfd8146a50 0x7fcfd80cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58790/0 (socket says 172.21.15.111:58790) 2023-12-04T05:39:24.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfd77fe640 1 -- 172.21.15.111:0/578161584 learned_addr learned my addr 172.21.15.111:0/578161584 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890326527 0 0) 0x7fcfd80cadf0 con 0x7fcfd8146a50 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc0003520 con 0x7fcfd8146a50 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.880+0000 7fcfd7fff640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfd806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31357370 0 0) 0x7fcfc0003520 con 0x7fcfd8146a50 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 msgr2=0x7fcfd806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfd806be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 >> v1:172.21.15.192:6789/0 conn(0x7fcfd8161a80 legacy=0x7fcfd80b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80ce4e0 con 0x7fcfd8146a50 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fcfc8001690 con 0x7fcfd8146a50 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 >> v1:172.21.15.111:6789/0 conn(0x7fcfd8146a50 legacy=0x7fcfd80cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00039a0 0x7fcfc0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfc00047c0 0x7fcfc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8161f10 con 0x7fcfc00047c0 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc000bae0 con 0x7fcfc00039a0 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc000bca0 con 0x7fcfd80b32b0 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd6ffd640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfc00047c0 0x7fcfc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd77fe640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00039a0 0x7fcfc0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd7fff640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd6ffd640 1 -- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 msgr2=0x7fcfc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd6ffd640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd6ffd640 1 -- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00039a0 msgr2=0x7fcfc0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd6ffd640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00039a0 0x7fcfc0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd6ffd640 1 -- 172.21.15.111:0/578161584 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc0003520 con 0x7fcfc00047c0 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.881+0000 7fcfd7fff640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b32b0 0x7fcfc00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfd6ffd640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfc00047c0 0x7fcfc0004c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc001750 tx=0x7fcfc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b805237fd2bc03b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0052d0 con 0x7fcfc00047c0 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc02e050 con 0x7fcfc00047c0 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfd67fc640 1 -- 172.21.15.111:0/578161584 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0035f0 con 0x7fcfc00047c0 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfddfa9640 1 -- 172.21.15.111:0/578161584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfc00047c0 msgr2=0x7fcfc0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/578161584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfc00047c0 0x7fcfc0004c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc001750 tx=0x7fcfc00057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfd7fff640 1 -- 172.21.15.111:0/578161584 reap_dead start 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfddfa9640 1 -- 172.21.15.111:0/578161584 shutdown_connections 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfddfa9640 1 -- 172.21.15.111:0/578161584 >> 172.21.15.111:0/578161584 conn(0x7fcfd806a4c0 msgr2=0x7fcfd806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfddfa9640 1 -- 172.21.15.111:0/578161584 shutdown_connections 2023-12-04T05:39:24.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.882+0000 7fcfddfa9640 1 -- 172.21.15.111:0/578161584 wait complete. 2023-12-04T05:39:24.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.883+0000 7fcfddfa9640 1 Processor -- start 2023-12-04T05:39:24.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 -- start start 2023-12-04T05:39:24.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00047c0 0x7fcfd80b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b2100 0x7fcfd8160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 0x7fcfd80b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8140100 con 0x7fcfd81411a0 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81403e0 con 0x7fcfc00047c0 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfddfa9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81406c0 con 0x7fcfd80b2100 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 0x7fcfd80b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 0x7fcfd80b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59098/0 (socket says 172.21.15.111:59098) 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.884+0000 7fcfd7fff640 1 -- 172.21.15.111:0/2536358927 learned_addr learned my addr 172.21.15.111:0/2536358927 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd77fe640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00047c0 0x7fcfd80b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd6ffd640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b2100 0x7fcfd8160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd7fff640 1 -- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b2100 msgr2=0x7fcfd8160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd7fff640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b2100 0x7fcfd8160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd7fff640 1 -- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00047c0 msgr2=0x7fcfd80b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd7fff640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00047c0 0x7fcfd80b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:24.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd7fff640 1 -- 172.21.15.111:0/2536358927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd81409a0 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfd7fff640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 0x7fcfd80b3fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8071dd0 tx=0x7fcfd0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd001b070 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd8140c80 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd000f040 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000e050 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.885+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd813fd60 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.886+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8140e70 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.886+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fcfd0015440 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.887+0000 7fcfbffff640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcfc406b2d0 0x7fcfc406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.887+0000 7fcfd77fe640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcfc406b2d0 0x7fcfc406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.887+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfd0089fe0 con 0x7fcfd81411a0 2023-12-04T05:39:24.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.887+0000 7fcfd77fe640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcfc406b2d0 0x7fcfc406d790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8001050 tx=0x7fcfc8001c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:24.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:24.891+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfd0054e80 con 0x7fcfd81411a0 2023-12-04T05:39:25.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.092+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcfd80693d0 con 0x7fcfd81411a0 2023-12-04T05:39:25.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.093+0000 7fcfbffff640 1 -- 172.21.15.111:0/2536358927 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcfd00584a0 con 0x7fcfd81411a0 2023-12-04T05:39:25.094 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:25.094 INFO:teuthology.orchestra.run.smithi111.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcfc406b2d0 msgr2=0x7fcfc406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcfc406b2d0 0x7fcfc406d790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8001050 tx=0x7fcfc8001c70 comp rx=0 tx=0).stop 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 msgr2=0x7fcfd80b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 0x7fcfd80b3fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8071dd0 tx=0x7fcfd0009f90 comp rx=0 tx=0).stop 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 shutdown_connections 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcfc406b2d0 0x7fcfc406d790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfd81411a0 0x7fcfd80b3fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfd80b2100 0x7fcfd8160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.094+0000 7fcfddfa9640 1 --2- 172.21.15.111:0/2536358927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc00047c0 0x7fcfd80b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.095+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 >> 172.21.15.111:0/2536358927 conn(0x7fcfd806a4c0 msgr2=0x7fcfd80d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.095+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 shutdown_connections 2023-12-04T05:39:25.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.095+0000 7fcfddfa9640 1 -- 172.21.15.111:0/2536358927 wait complete. 2023-12-04T05:39:25.107 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-04T05:39:25.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:25.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.205+0000 7f8bdf93c640 1 Processor -- start 2023-12-04T05:39:25.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.206+0000 7f8bdf93c640 1 -- start start 2023-12-04T05:39:25.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.206+0000 7f8bdf93c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8bd815d210 0x7f8bd806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.206+0000 7f8bdf93c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8bd80af4f0 0x7f8bd80b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.206+0000 7f8bdf93c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd8161d00 con 0x7f8bd806e520 2023-12-04T05:39:25.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.206+0000 7f8bdf93c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd8161180 con 0x7f8bd80af4f0 2023-12-04T05:39:25.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.206+0000 7f8bdf93c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd8161460 con 0x7f8bd815d210 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdced7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8bd806e520 0x7f8bd80aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58806/0 (socket says 172.21.15.111:58806) 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdced7640 1 -- 172.21.15.111:0/658017952 learned_addr learned my addr 172.21.15.111:0/658017952 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdd6d8640 1 --2- 172.21.15.111:0/658017952 >> v2:172.21.15.192:3300/0 conn(0x7f8bd815d210 0x7f8bd806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdded9640 1 --2- 172.21.15.111:0/658017952 >> v2:172.21.15.111:3300/0 conn(0x7f8bd80af4f0 0x7f8bd80b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bcbfff640 1 -- 172.21.15.111:0/658017952 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479180094 0 0) 0x7f8bd8161d00 con 0x7f8bd806e520 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bcbfff640 1 -- 172.21.15.111:0/658017952 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc4003520 con 0x7f8bd806e520 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdd6d8640 1 -- 172.21.15.111:0/658017952 >> v2:172.21.15.111:3300/0 conn(0x7f8bd80af4f0 msgr2=0x7f8bd80b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdd6d8640 1 --2- 172.21.15.111:0/658017952 >> v2:172.21.15.111:3300/0 conn(0x7f8bd80af4f0 0x7f8bd80b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdd6d8640 1 -- 172.21.15.111:0/658017952 >> v1:172.21.15.111:6789/0 conn(0x7f8bd806e520 legacy=0x7f8bd80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.207+0000 7f8bdd6d8640 1 -- 172.21.15.111:0/658017952 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd8161740 con 0x7f8bd815d210 2023-12-04T05:39:25.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.208+0000 7f8bdd6d8640 1 --2- 172.21.15.111:0/658017952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd815d210 0x7f8bd806dfe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8bd806b110 tx=0x7f8bd002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c9c5a72c7947ed server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.208+0000 7f8bcbfff640 1 -- 172.21.15.111:0/658017952 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd002e070 con 0x7f8bd815d210 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.208+0000 7f8bcbfff640 1 -- 172.21.15.111:0/658017952 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd00315a0 con 0x7f8bd815d210 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.208+0000 7f8bcbfff640 1 -- 172.21.15.111:0/658017952 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0031910 con 0x7f8bd815d210 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 -- 172.21.15.111:0/658017952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd815d210 msgr2=0x7f8bd806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/658017952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd815d210 0x7f8bd806dfe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8bd806b110 tx=0x7f8bd002b840 comp rx=0 tx=0).stop 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 -- 172.21.15.111:0/658017952 shutdown_connections 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/658017952 >> v2:172.21.15.111:3300/0 conn(0x7f8bd80af4f0 0x7f8bd80b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/658017952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd815d210 0x7f8bd806dfe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 -- 172.21.15.111:0/658017952 >> 172.21.15.111:0/658017952 conn(0x7f8bd80b1c00 msgr2=0x7f8bd80b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 -- 172.21.15.111:0/658017952 shutdown_connections 2023-12-04T05:39:25.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.209+0000 7f8bdf93c640 1 -- 172.21.15.111:0/658017952 wait complete. 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.210+0000 7f8bdf93c640 1 Processor -- start 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.210+0000 7f8bdf93c640 1 -- start start 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdf93c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd806e520 0x7f8bd8148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdf93c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bd80af4f0 0x7f8bd815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdf93c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 0x7f8bd80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdf93c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8148ea0 con 0x7f8bd815d210 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdf93c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd815e1e0 con 0x7f8bd806e520 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdf93c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80cd070 con 0x7f8bd80af4f0 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 0x7f8bd80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 0x7f8bd80b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59120/0 (socket says 172.21.15.111:59120) 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 -- 172.21.15.111:0/257408705 learned_addr learned my addr 172.21.15.111:0/257408705 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdd6d8640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd806e520 0x7f8bd8148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdced7640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bd80af4f0 0x7f8bd815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 -- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bd80af4f0 msgr2=0x7f8bd815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bd80af4f0 0x7f8bd815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 -- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd806e520 msgr2=0x7f8bd8148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd806e520 0x7f8bd8148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 -- 172.21.15.111:0/257408705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009400 con 0x7f8bd815d210 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.211+0000 7f8bdded9640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 0x7f8bd80b06b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400efc0 tx=0x7f8bd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.212+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4009ca0 con 0x7f8bd815d210 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.212+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd400f040 con 0x7f8bd815d210 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.212+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40146c0 con 0x7f8bd815d210 2023-12-04T05:39:25.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.212+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd80cd350 con 0x7f8bd815d210 2023-12-04T05:39:25.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.212+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd80cd910 con 0x7f8bd815d210 2023-12-04T05:39:25.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.213+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd80cdb30 con 0x7f8bd815d210 2023-12-04T05:39:25.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.213+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8bd4014820 con 0x7f8bd815d210 2023-12-04T05:39:25.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.214+0000 7f8bc9ffb640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8bbc06b2a0 0x7f8bbc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.214+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bd408a9d0 con 0x7f8bd815d210 2023-12-04T05:39:25.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.214+0000 7f8bdd6d8640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8bbc06b2a0 0x7f8bbc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.214+0000 7f8bdd6d8640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8bbc06b2a0 0x7f8bbc06d760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0036000 tx=0x7f8bd003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.218+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bd40558f0 con 0x7f8bd815d210 2023-12-04T05:39:25.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.386+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bd8067460 con 0x7f8bd815d210 2023-12-04T05:39:25.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.387+0000 7f8bc9ffb640 1 -- 172.21.15.111:0/257408705 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f8bd4018090 con 0x7f8bd815d210 2023-12-04T05:39:25.388 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:25.388 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:25.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8bbc06b2a0 msgr2=0x7f8bbc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8bbc06b2a0 0x7f8bbc06d760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0036000 tx=0x7f8bd003d040 comp rx=0 tx=0).stop 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 msgr2=0x7f8bd80b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 0x7f8bd80b06b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400efc0 tx=0x7f8bd400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 shutdown_connections 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8bbc06b2a0 0x7f8bbc06d760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8bd815d210 0x7f8bd80b06b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bd80af4f0 0x7f8bd815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 --2- 172.21.15.111:0/257408705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bd806e520 0x7f8bd8148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 >> 172.21.15.111:0/257408705 conn(0x7f8bd80b1c00 msgr2=0x7f8bd80b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 shutdown_connections 2023-12-04T05:39:25.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.389+0000 7f8bdf93c640 1 -- 172.21.15.111:0/257408705 wait complete. 2023-12-04T05:39:25.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:25.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.502+0000 7f1ee3e2b640 1 Processor -- start 2023-12-04T05:39:25.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.502+0000 7f1ee3e2b640 1 -- start start 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee3e2b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1edc06bbe0 0x7f1edc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee3e2b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0b4780 con 0x7f1edc15d110 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee3e2b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc06d970 con 0x7f1edc06d520 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee3e2b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1edc0ad5d0 con 0x7f1edc06bbe0 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee1bc7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1edc15d110 0x7f1edc06b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58820/0 (socket says 172.21.15.111:58820) 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee1bc7640 1 -- 172.21.15.111:0/1541568855 learned_addr learned my addr 172.21.15.111:0/1541568855 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee13c6640 1 --2- 172.21.15.111:0/1541568855 >> v2:172.21.15.192:3300/0 conn(0x7f1edc06bbe0 0x7f1edc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158770796 0 0) 0x7f1edc0b4780 con 0x7f1edc15d110 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.503+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec8003520 con 0x7f1edc15d110 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300781052 0 0) 0x7f1ec8003520 con 0x7f1edc15d110 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 >> v2:172.21.15.192:3300/0 conn(0x7f1edc06bbe0 msgr2=0x7f1edc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 --2- 172.21.15.111:0/1541568855 >> v2:172.21.15.192:3300/0 conn(0x7f1edc06bbe0 0x7f1edc15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 >> v1:172.21.15.192:6789/0 conn(0x7f1edc06d520 legacy=0x7f1edc0ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0b41c0 con 0x7f1edc15d110 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f1ed8001a10 con 0x7f1edc15d110 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 >> v1:172.21.15.111:6789/0 conn(0x7f1edc15d110 legacy=0x7f1edc06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc0ad5d0 con 0x7f1ec80047c0 2023-12-04T05:39:25.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec800c020 con 0x7f1ec8005180 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec800c1e0 con 0x7f1ec8003970 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee13c6640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee1bc7640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee23c8640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee13c6640 1 -- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8003970 msgr2=0x7f1ec8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee13c6640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee13c6640 1 -- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8005180 msgr2=0x7f1ec800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee13c6640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee13c6640 1 -- 172.21.15.111:0/1541568855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8003520 con 0x7f1ec80047c0 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee1bc7640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8003970 0x7f1ec8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee23c8640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8005180 0x7f1ec800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:25.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.504+0000 7f1ee23c8640 1 -- 172.21.15.111:0/1541568855 reap_dead start 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee13c6640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009ac0 tx=0x7f1ec80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79201c7e71e253b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc02c6b0 con 0x7f1ec80047c0 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc02c810 con 0x7f1ec80047c0 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee0bc5640 1 -- 172.21.15.111:0/1541568855 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc02cba0 con 0x7f1ec80047c0 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/1541568855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 msgr2=0x7f1ec8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009ac0 tx=0x7f1ec80057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/1541568855 shutdown_connections 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/1541568855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ec80047c0 0x7f1ec8004c40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.505+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/1541568855 >> 172.21.15.111:0/1541568855 conn(0x7f1edc0b1b90 msgr2=0x7f1edc070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.506+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/1541568855 shutdown_connections 2023-12-04T05:39:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.506+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/1541568855 wait complete. 2023-12-04T05:39:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 Processor -- start 2023-12-04T05:39:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 -- start start 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec80047c0 0x7f1edc13b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1e65c0 0x7f1edc14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 0x7f1edc14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc1529c0 con 0x7f1edc14e7d0 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc13b700 con 0x7f1ec80047c0 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.507+0000 7f1ee3e2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc1e6b30 con 0x7f1edc1e65c0 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 0x7f1edc14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 0x7f1edc14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59136/0 (socket says 172.21.15.111:59136) 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee1bc7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec80047c0 0x7f1edc13b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 -- 172.21.15.111:0/3351554444 learned_addr learned my addr 172.21.15.111:0/3351554444 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:25.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee13c6640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1e65c0 0x7f1edc14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 -- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1e65c0 msgr2=0x7f1edc14cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1e65c0 0x7f1edc14cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 -- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec80047c0 msgr2=0x7f1edc13b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec80047c0 0x7f1edc13b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 -- 172.21.15.111:0/3351554444 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc1e6e10 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee23c8640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 0x7f1edc14ec50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1edc071810 tx=0x7f1ed400e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40050d0 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4005230 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc1e70f0 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40035f0 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.508+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1edc14d4f0 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.509+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1edc1e7610 con 0x7f1edc14e7d0 2023-12-04T05:39:25.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.509+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1ed4003750 con 0x7f1edc14e7d0 2023-12-04T05:39:25.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.510+0000 7f1ed27fc640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec006b2a0 0x7f1ec006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.510+0000 7f1ee1bc7640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec006b2a0 0x7f1ec006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.510+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ed4089010 con 0x7f1edc14e7d0 2023-12-04T05:39:25.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.510+0000 7f1ee1bc7640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec006b2a0 0x7f1ec006d760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001b40 tx=0x7f1ed8001890 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.514+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ed4053f30 con 0x7f1edc14e7d0 2023-12-04T05:39:25.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.680+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1edc069360 con 0x7f1edc14e7d0 2023-12-04T05:39:25.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.681+0000 7f1ed27fc640 1 -- 172.21.15.111:0/3351554444 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ed40190a0 con 0x7f1edc14e7d0 2023-12-04T05:39:25.683 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:25.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec006b2a0 msgr2=0x7f1ec006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec006b2a0 0x7f1ec006d760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001b40 tx=0x7f1ed8001890 comp rx=0 tx=0).stop 2023-12-04T05:39:25.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 msgr2=0x7f1edc14ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 0x7f1edc14ec50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1edc071810 tx=0x7f1ed400e5f0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 shutdown_connections 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec006b2a0 0x7f1ec006d760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1edc14e7d0 0x7f1edc14ec50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1e65c0 0x7f1edc14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 --2- 172.21.15.111:0/3351554444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec80047c0 0x7f1edc13b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 >> 172.21.15.111:0/3351554444 conn(0x7f1edc0b1b90 msgr2=0x7f1edc14d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 shutdown_connections 2023-12-04T05:39:25.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.683+0000 7f1ee3e2b640 1 -- 172.21.15.111:0/3351554444 wait complete. 2023-12-04T05:39:25.695 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:25.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.796+0000 7fbed1b7c640 1 Processor -- start 2023-12-04T05:39:25.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.796+0000 7fbed1b7c640 1 -- start start 2023-12-04T05:39:25.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbed1b7c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc146b10 0x7fbecc161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbed1b7c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbecc0d0f10 0x7fbecc0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbed1b7c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0d18b0 con 0x7fbecc15d8a0 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbed1b7c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc06eaf0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbed1b7c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc15e4e0 con 0x7fbecc146b10 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbecc0d0f10 0x7fbecc0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbecc0d0f10 0x7fbecc0d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59138/0 (socket says 172.21.15.111:59138) 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 -- 172.21.15.111:0/505303098 learned_addr learned my addr 172.21.15.111:0/505303098 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecb7fe640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc146b10 0x7fbecc161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 -- 172.21.15.111:0/505303098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc146b10 msgr2=0x7fbecc161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc146b10 0x7fbecc161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 -- 172.21.15.111:0/505303098 >> v1:172.21.15.192:6789/0 conn(0x7fbecc15d8a0 legacy=0x7fbecc15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecaffd640 1 -- 172.21.15.111:0/505303098 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc15e7c0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.797+0000 7fbecb7fe640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc146b10 0x7fbecc161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:25.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.798+0000 7fbecaffd640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0d1370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbec00098a0 tx=0x7fbec002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e1f57fe02ed56 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.798+0000 7fbeca7fc640 1 -- 172.21.15.111:0/505303098 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0031640 con 0x7fbecc0d0f10 2023-12-04T05:39:25.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.798+0000 7fbeca7fc640 1 -- 172.21.15.111:0/505303098 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec00317a0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.798+0000 7fbeca7fc640 1 -- 172.21.15.111:0/505303098 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0031b30 con 0x7fbecc0d0f10 2023-12-04T05:39:25.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 -- 172.21.15.111:0/505303098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 msgr2=0x7fbecc0d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0d1370 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbec00098a0 tx=0x7fbec002b8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 -- 172.21.15.111:0/505303098 shutdown_connections 2023-12-04T05:39:25.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0d1370 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/505303098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc146b10 0x7fbecc161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 -- 172.21.15.111:0/505303098 >> 172.21.15.111:0/505303098 conn(0x7fbecc070af0 msgr2=0x7fbecc0662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 -- 172.21.15.111:0/505303098 shutdown_connections 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.799+0000 7fbed1b7c640 1 -- 172.21.15.111:0/505303098 wait complete. 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.800+0000 7fbed1b7c640 1 Processor -- start 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.800+0000 7fbed1b7c640 1 -- start start 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbed1b7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbed1b7c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbecc146b10 0x7fbecc1420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbed1b7c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc15d8a0 0x7fbecc142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbed1b7c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0cdbe0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbed1b7c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0cdec0 con 0x7fbecc146b10 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbed1b7c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0ce1a0 con 0x7fbecc15d8a0 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59148/0 (socket says 172.21.15.111:59148) 2023-12-04T05:39:25.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 -- 172.21.15.111:0/1319382241 learned_addr learned my addr 172.21.15.111:0/1319382241 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:25.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecaffd640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbecc146b10 0x7fbecc1420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecbfff640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc15d8a0 0x7fbecc142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 -- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc15d8a0 msgr2=0x7fbecc142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc15d8a0 0x7fbecc142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 -- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbecc146b10 msgr2=0x7fbecc1420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbecc146b10 0x7fbecc1420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 -- 172.21.15.111:0/1319382241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00094d0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbecb7fe640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0cf920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c7f0 tx=0x7fbebc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007a30 con 0x7fbecc0d0f10 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.801+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc007b90 con 0x7fbecc0d0f10 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.802+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00ce40 con 0x7fbecc0d0f10 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.802+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc0ce480 con 0x7fbecc0d0f10 2023-12-04T05:39:25.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.802+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc0cfe60 con 0x7fbecc0d0f10 2023-12-04T05:39:25.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.802+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc0d0050 con 0x7fbecc0d0f10 2023-12-04T05:39:25.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.803+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbebc0113f0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.803+0000 7fbeb3fff640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbeb806b2a0 0x7fbeb806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:25.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.803+0000 7fbecaffd640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbeb806b2a0 0x7fbeb806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:25.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.803+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbebc089d70 con 0x7fbecc0d0f10 2023-12-04T05:39:25.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.804+0000 7fbecaffd640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbeb806b2a0 0x7fbeb806d760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbec0036000 tx=0x7fbec003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:25.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.808+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbebc054be0 con 0x7fbecc0d0f10 2023-12-04T05:39:25.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.976+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbecc069360 con 0x7fbecc0d0f10 2023-12-04T05:39:25.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.976+0000 7fbeb3fff640 1 -- 172.21.15.111:0/1319382241 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbebc01f020 con 0x7fbecc0d0f10 2023-12-04T05:39:25.978 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:25.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbeb806b2a0 msgr2=0x7fbeb806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbeb806b2a0 0x7fbeb806d760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbec0036000 tx=0x7fbec003c040 comp rx=0 tx=0).stop 2023-12-04T05:39:25.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 msgr2=0x7fbecc0cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:25.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0cf920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c7f0 tx=0x7fbebc00cbf0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 shutdown_connections 2023-12-04T05:39:25.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbeb806b2a0 0x7fbeb806d760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbecc15d8a0 0x7fbecc142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbecc146b10 0x7fbecc1420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 --2- 172.21.15.111:0/1319382241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbecc0d0f10 0x7fbecc0cf920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:25.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 >> 172.21.15.111:0/1319382241 conn(0x7fbecc070af0 msgr2=0x7fbecc0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:25.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 shutdown_connections 2023-12-04T05:39:25.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:25.978+0000 7fbed1b7c640 1 -- 172.21.15.111:0/1319382241 wait complete. 2023-12-04T05:39:25.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:26.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.091+0000 7f20787c9640 1 Processor -- start 2023-12-04T05:39:26.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.091+0000 7f20787c9640 1 -- start start 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f20787c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207006bbe0 0x7f207006d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f20787c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20700b3a20 0x7f20700b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f20787c9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20700b43f0 con 0x7f2070160920 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f20787c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700b3850 con 0x7f20700b3a20 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f20787c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20700b46a0 con 0x7f207006bbe0 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2076565640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2070160920 0x7f207006b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58824/0 (socket says 172.21.15.111:58824) 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2076565640 1 -- 172.21.15.111:0/1475417794 learned_addr learned my addr 172.21.15.111:0/1475417794 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075d64640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207006bbe0 0x7f207006d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2076d66640 1 --2- 172.21.15.111:0/1475417794 >> v2:172.21.15.192:3300/0 conn(0x7f20700b3a20 0x7f20700b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468299661 0 0) 0x7f20700b43f0 con 0x7f2070160920 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205c003520 con 0x7f2070160920 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 598307363 0 0) 0x7f205c003520 con 0x7f2070160920 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207006bbe0 msgr2=0x7f207006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207006bbe0 0x7f207006d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 >> v2:172.21.15.192:3300/0 conn(0x7f20700b3a20 msgr2=0x7f20700b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 --2- 172.21.15.111:0/1475417794 >> v2:172.21.15.192:3300/0 conn(0x7f20700b3a20 0x7f20700b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207006cb60 con 0x7f2070160920 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f206c001670 con 0x7f2070160920 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2076d66640 1 --2- 172.21.15.111:0/1475417794 >> v2:172.21.15.192:3300/0 conn(0x7f20700b3a20 0x7f20700b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 >> v1:172.21.15.111:6789/0 conn(0x7f2070160920 legacy=0x7f207006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f205c003970 0x7f205c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f205c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f205c005180 0x7f205c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20700b46a0 con 0x7f205c0047c0 2023-12-04T05:39:26.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00bec0 con 0x7f205c003970 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00c000 con 0x7f205c005180 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.092+0000 7f2075d64640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207006bbe0 0x7f207006d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f205c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2076565640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f205c003970 0x7f205c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2076d66640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f205c005180 0x7f205c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 -- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f205c005180 msgr2=0x7f205c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f205c005180 0x7f205c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 -- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f205c003970 msgr2=0x7f205c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f205c003970 0x7f205c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 -- 172.21.15.111:0/1475417794 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c003520 con 0x7f205c0047c0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.093+0000 7f2075d64640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f205c004c40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2060009b00 tx=0x7f205c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9e23811be40591 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060004030 con 0x7f205c0047c0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060004190 con 0x7f205c0047c0 2023-12-04T05:39:26.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f2075563640 1 -- 172.21.15.111:0/1475417794 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060004520 con 0x7f205c0047c0 2023-12-04T05:39:26.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f20787c9640 1 -- 172.21.15.111:0/1475417794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 msgr2=0x7f205c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f20787c9640 1 --2- 172.21.15.111:0/1475417794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f205c004c40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2060009b00 tx=0x7f205c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f2076d66640 1 -- 172.21.15.111:0/1475417794 reap_dead start 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f20787c9640 1 -- 172.21.15.111:0/1475417794 shutdown_connections 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f20787c9640 1 -- 172.21.15.111:0/1475417794 >> 172.21.15.111:0/1475417794 conn(0x7f20700b1b90 msgr2=0x7f2070070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f20787c9640 1 -- 172.21.15.111:0/1475417794 shutdown_connections 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.094+0000 7f20787c9640 1 -- 172.21.15.111:0/1475417794 wait complete. 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.095+0000 7f20787c9640 1 Processor -- start 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.095+0000 7f20787c9640 1 -- start start 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f20787c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f20700af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f20787c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2070145f70 0x7f20701463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f20787c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2070157410 0x7f20701578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f20787c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20701583e0 con 0x7f205c0047c0 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f20787c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2070158780 con 0x7f2070145f70 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f20787c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2070146930 con 0x7f2070157410 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f20700af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f20700af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59174/0 (socket says 172.21.15.111:59174) 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 -- 172.21.15.111:0/2831280900 learned_addr learned my addr 172.21.15.111:0/2831280900 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 -- 172.21.15.111:0/2831280900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2070157410 msgr2=0x7f20701578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2070157410 0x7f20701578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 -- 172.21.15.111:0/2831280900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2070145f70 msgr2=0x7f20701463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2070145f70 0x7f20701463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.096+0000 7f2076565640 1 -- 172.21.15.111:0/2831280900 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070158100 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.097+0000 7f2076565640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f20700af9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f206c007290 tx=0x7f206c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.097+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c007e10 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.097+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c00f960 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.097+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2070146df0 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.097+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00fd10 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.097+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20701473b0 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.098+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2070146c50 con 0x7f205c0047c0 2023-12-04T05:39:26.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.098+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f206c01d460 con 0x7f205c0047c0 2023-12-04T05:39:26.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.098+0000 7f2066ffd640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f205406b2a0 0x7f205406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.099+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f206c089b80 con 0x7f205c0047c0 2023-12-04T05:39:26.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.099+0000 7f2075d64640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f205406b2a0 0x7f205406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.099+0000 7f2075d64640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f205406b2a0 0x7f205406d760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2060035000 tx=0x7f206002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.103+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2070146c50 con 0x7f205c0047c0 2023-12-04T05:39:26.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.272+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2070146c50 con 0x7f205c0047c0 2023-12-04T05:39:26.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.272+0000 7f2066ffd640 1 -- 172.21.15.111:0/2831280900 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2070146c50 con 0x7f205c0047c0 2023-12-04T05:39:26.274 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:26.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f205406b2a0 msgr2=0x7f205406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f205406b2a0 0x7f205406d760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2060035000 tx=0x7f206002e000 comp rx=0 tx=0).stop 2023-12-04T05:39:26.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 msgr2=0x7f20700af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f20700af9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f206c007290 tx=0x7f206c007750 comp rx=0 tx=0).stop 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 shutdown_connections 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f205406b2a0 0x7f205406d760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2070157410 0x7f20701578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2070145f70 0x7f20701463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 --2- 172.21.15.111:0/2831280900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f205c0047c0 0x7f20700af9d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 >> 172.21.15.111:0/2831280900 conn(0x7f20700b1b90 msgr2=0x7f207015f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 shutdown_connections 2023-12-04T05:39:26.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.274+0000 7f20787c9640 1 -- 172.21.15.111:0/2831280900 wait complete. 2023-12-04T05:39:26.287 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:26.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.384+0000 7f35d2ecf640 1 Processor -- start 2023-12-04T05:39:26.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.384+0000 7f35d2ecf640 1 -- start start 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d2ecf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f35cc06b4d0 0x7f35cc06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d2ecf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc0add00 0x7f35cc0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d2ecf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc06be70 con 0x7f35cc1544c0 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d2ecf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc0ad5d0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d2ecf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc06df90 con 0x7f35cc0add00 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f35cc06b4d0 0x7f35cc06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f35cc06b4d0 0x7f35cc06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59178/0 (socket says 172.21.15.111:59178) 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 -- 172.21.15.111:0/3663023559 learned_addr learned my addr 172.21.15.111:0/3663023559 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d146c640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc0add00 0x7f35cc0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 -- 172.21.15.111:0/3663023559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc0add00 msgr2=0x7f35cc0ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc0add00 0x7f35cc0ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 -- 172.21.15.111:0/3663023559 >> v1:172.21.15.192:6789/0 conn(0x7f35cc1544c0 legacy=0x7f35cc06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35cbfff640 1 -- 172.21.15.111:0/3663023559 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc1548a0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.385+0000 7f35d146c640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc0add00 0x7f35cc0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35cbfff640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc06b930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009870 tx=0x7f35b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c29820561c12f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35cb7fe640 1 -- 172.21.15.111:0/3663023559 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8031640 con 0x7f35cc06b4d0 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35cb7fe640 1 -- 172.21.15.111:0/3663023559 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b80317a0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35cb7fe640 1 -- 172.21.15.111:0/3663023559 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8031b30 con 0x7f35cc06b4d0 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 -- 172.21.15.111:0/3663023559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 msgr2=0x7f35cc06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc06b930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009870 tx=0x7f35b802b8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 -- 172.21.15.111:0/3663023559 shutdown_connections 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc0add00 0x7f35cc0ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/3663023559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc06b930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 -- 172.21.15.111:0/3663023559 >> 172.21.15.111:0/3663023559 conn(0x7f35cc0b1b50 msgr2=0x7f35cc070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:26.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 -- 172.21.15.111:0/3663023559 shutdown_connections 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.386+0000 7f35d2ecf640 1 -- 172.21.15.111:0/3663023559 wait complete. 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.388+0000 7f35d2ecf640 1 Processor -- start 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.388+0000 7f35d2ecf640 1 -- start start 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35cc0add00 0x7f35cc141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc1544c0 0x7f35cc1f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc14e110 con 0x7f35cc06b4d0 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc14e3f0 con 0x7f35cc0add00 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc14e6d0 con 0x7f35cc1544c0 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc0ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59192/0 (socket says 172.21.15.111:59192) 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 -- 172.21.15.111:0/2565241725 learned_addr learned my addr 172.21.15.111:0/2565241725 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d146c640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc1544c0 0x7f35cc1f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35cbfff640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35cc0add00 0x7f35cc141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 -- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc1544c0 msgr2=0x7f35cc1f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc1544c0 0x7f35cc1f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 -- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35cc0add00 msgr2=0x7f35cc141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35cc0add00 0x7f35cc141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 -- 172.21.15.111:0/2565241725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80094d0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d0c6b640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc0ceaa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c430 tx=0x7f35c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0011790 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c00118f0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc152c40 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0011ce0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.389+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc142400 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.390+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc0693d0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.391+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f35c001c460 con 0x7f35cc06b4d0 2023-12-04T05:39:26.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.391+0000 7f35c97fa640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f35b006b2a0 0x7f35b006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.391+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35c0089b90 con 0x7f35cc06b4d0 2023-12-04T05:39:26.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.391+0000 7f35cbfff640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f35b006b2a0 0x7f35b006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.392+0000 7f35cbfff640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f35b006b2a0 0x7f35b006d760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35b8036000 tx=0x7f35b803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.395+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35c0054a00 con 0x7f35cc06b4d0 2023-12-04T05:39:26.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.597+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35cc071bf0 con 0x7f35cc06b4d0 2023-12-04T05:39:26.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.598+0000 7f35c97fa640 1 -- 172.21.15.111:0/2565241725 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f35c0058020 con 0x7f35cc06b4d0 2023-12-04T05:39:26.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:26.599 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:26.599 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:26.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.599+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f35b006b2a0 msgr2=0x7f35b006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.599+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f35b006b2a0 0x7f35b006d760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35b8036000 tx=0x7f35b803c040 comp rx=0 tx=0).stop 2023-12-04T05:39:26.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 msgr2=0x7f35cc0ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc0ceaa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c430 tx=0x7f35c000c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 shutdown_connections 2023-12-04T05:39:26.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f35b006b2a0 0x7f35b006d760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35cc1544c0 0x7f35cc1f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35cc0add00 0x7f35cc141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 --2- 172.21.15.111:0/2565241725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35cc06b4d0 0x7f35cc0ceaa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 >> 172.21.15.111:0/2565241725 conn(0x7f35cc0b1b50 msgr2=0x7f35cc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:26.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 shutdown_connections 2023-12-04T05:39:26.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.600+0000 7f35d2ecf640 1 -- 172.21.15.111:0/2565241725 wait complete. 2023-12-04T05:39:26.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:26.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 Processor -- start 2023-12-04T05:39:26.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 -- start start 2023-12-04T05:39:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa60413cee0 0x7fa60406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa604140200 0x7fa60406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040aea20 con 0x7fa60415a490 2023-12-04T05:39:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60413d2c0 con 0x7fa604140200 2023-12-04T05:39:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.711+0000 7fa60a7c5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60415b3a0 con 0x7fa60413cee0 2023-12-04T05:39:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa6037fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa604140200 0x7fa60406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa608d62640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa60415a490 0x7fa60415a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58836/0 (socket says 172.21.15.111:58836) 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa608d62640 1 -- 172.21.15.111:0/2280771480 learned_addr learned my addr 172.21.15.111:0/2280771480 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa603fff640 1 --2- 172.21.15.111:0/2280771480 >> v2:172.21.15.192:3300/0 conn(0x7fa60413cee0 0x7fa60406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa6037fe640 1 -- 172.21.15.111:0/2280771480 >> v2:172.21.15.192:3300/0 conn(0x7fa60413cee0 msgr2=0x7fa60406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa6037fe640 1 --2- 172.21.15.111:0/2280771480 >> v2:172.21.15.192:3300/0 conn(0x7fa60413cee0 0x7fa60406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa6037fe640 1 -- 172.21.15.111:0/2280771480 >> v1:172.21.15.111:6789/0 conn(0x7fa60415a490 legacy=0x7fa60415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa6037fe640 1 -- 172.21.15.111:0/2280771480 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60415b0c0 con 0x7fa604140200 2023-12-04T05:39:26.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.712+0000 7fa602ffd640 1 -- 172.21.15.111:0/2280771480 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559487800 0 0) 0x7fa6040aea20 con 0x7fa60415a490 2023-12-04T05:39:26.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa6037fe640 1 --2- 172.21.15.111:0/2280771480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa604140200 0x7fa60406c790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6040b2280 tx=0x7fa5f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb83a4b47159bd64 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa602ffd640 1 -- 172.21.15.111:0/2280771480 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f802e070 con 0x7fa604140200 2023-12-04T05:39:26.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa602ffd640 1 -- 172.21.15.111:0/2280771480 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f80315a0 con 0x7fa604140200 2023-12-04T05:39:26.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa602ffd640 1 -- 172.21.15.111:0/2280771480 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8031930 con 0x7fa604140200 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 -- 172.21.15.111:0/2280771480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa604140200 msgr2=0x7fa60406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/2280771480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa604140200 0x7fa60406c790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6040b2280 tx=0x7fa5f802b840 comp rx=0 tx=0).stop 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 -- 172.21.15.111:0/2280771480 shutdown_connections 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/2280771480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa604140200 0x7fa60406c790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/2280771480 >> v2:172.21.15.192:3300/0 conn(0x7fa60413cee0 0x7fa60406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 -- 172.21.15.111:0/2280771480 >> 172.21.15.111:0/2280771480 conn(0x7fa60406a4c0 msgr2=0x7fa60406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 -- 172.21.15.111:0/2280771480 shutdown_connections 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.713+0000 7fa60a7c5640 1 -- 172.21.15.111:0/2280771480 wait complete. 2023-12-04T05:39:26.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.714+0000 7fa60a7c5640 1 Processor -- start 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 -- start start 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60413cee0 0x7fa604136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa604140200 0x7fa6041f0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 0x7fa6040b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa604136d70 con 0x7fa60415a490 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041f0dc0 con 0x7fa60413cee0 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa60a7c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040b4490 con 0x7fa604140200 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa608d62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 0x7fa6040b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa608d62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 0x7fa6040b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59198/0 (socket says 172.21.15.111:59198) 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa608d62640 1 -- 172.21.15.111:0/3622771080 learned_addr learned my addr 172.21.15.111:0/3622771080 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa603fff640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60413cee0 0x7fa604136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.715+0000 7fa6037fe640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa604140200 0x7fa6041f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa608d62640 1 -- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa604140200 msgr2=0x7fa6041f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa608d62640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa604140200 0x7fa6041f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa608d62640 1 -- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60413cee0 msgr2=0x7fa604136830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa608d62640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60413cee0 0x7fa604136830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa608d62640 1 -- 172.21.15.111:0/3622771080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009400 con 0x7fa60415a490 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa608d62640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 0x7fa6040b3f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c7b0 tx=0x7fa5fc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00a640 con 0x7fa60415a490 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00a7c0 con 0x7fa60415a490 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa604070620 con 0x7fa60415a490 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00ab70 con 0x7fa60415a490 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.716+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6041f12d0 con 0x7fa60415a490 2023-12-04T05:39:26.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.717+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040703f0 con 0x7fa60415a490 2023-12-04T05:39:26.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.717+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa5fc00acd0 con 0x7fa60415a490 2023-12-04T05:39:26.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.718+0000 7fa600ff9640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa5f006b2a0 0x7fa5f006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:26.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.718+0000 7fa603fff640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa5f006b2a0 0x7fa5f006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:26.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.718+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5fc089260 con 0x7fa60415a490 2023-12-04T05:39:26.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.718+0000 7fa603fff640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa5f006b2a0 0x7fa5f006d760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa6040cc830 tx=0x7fa5f4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:26.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.722+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6040703f0 con 0x7fa60415a490 2023-12-04T05:39:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.888+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6040703f0 con 0x7fa60415a490 2023-12-04T05:39:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.890+0000 7fa600ff9640 1 -- 172.21.15.111:0/3622771080 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fa6040703f0 con 0x7fa60415a490 2023-12-04T05:39:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:26.892 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa5f006b2a0 msgr2=0x7fa5f006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa5f006b2a0 0x7fa5f006d760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa6040cc830 tx=0x7fa5f4009190 comp rx=0 tx=0).stop 2023-12-04T05:39:26.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 msgr2=0x7fa6040b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:26.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 0x7fa6040b3f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c7b0 tx=0x7fa5fc00cc70 comp rx=0 tx=0).stop 2023-12-04T05:39:26.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 shutdown_connections 2023-12-04T05:39:26.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa5f006b2a0 0x7fa5f006d760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60415a490 0x7fa6040b3f50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa604140200 0x7fa6041f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 --2- 172.21.15.111:0/3622771080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60413cee0 0x7fa604136830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.891+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 >> 172.21.15.111:0/3622771080 conn(0x7fa60406a4c0 msgr2=0x7fa60415e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.892+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 shutdown_connections 2023-12-04T05:39:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:26.892+0000 7fa60a7c5640 1 -- 172.21.15.111:0/3622771080 wait complete. 2023-12-04T05:39:26.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:26.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:27.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.005+0000 7f538bec9640 1 Processor -- start 2023-12-04T05:39:27.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.005+0000 7f538bec9640 1 -- start start 2023-12-04T05:39:27.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538bec9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 0x7f53840cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538bec9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53840cfa10 con 0x7f53840b30e0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538bec9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53841406f0 con 0x7f53840cf0f0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538bec9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53840b4700 con 0x7f5384160e30 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538a466640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f53840b30e0 0x7f538406a4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58844/0 (socket says 172.21.15.111:58844) 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538a466640 1 -- 172.21.15.111:0/1595413712 learned_addr learned my addr 172.21.15.111:0/1595413712 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5389464640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 0x7f53840cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812209626 0 0) 0x7f53840cfa10 con 0x7f53840b30e0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5370003520 con 0x7f53840b30e0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430792887 0 0) 0x7f5370003520 con 0x7f53840b30e0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 msgr2=0x7f53840cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 0x7f53840cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 >> v1:172.21.15.192:6789/0 conn(0x7f53840cf0f0 legacy=0x7f5384143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384161360 con 0x7f53840b30e0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f537c0019f0 con 0x7f53840b30e0 2023-12-04T05:39:27.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 >> v1:172.21.15.111:6789/0 conn(0x7f53840b30e0 legacy=0x7f538406a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 0x7f53700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370003f20 0x7f53700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5370004d00 0x7f5370005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53840b4700 con 0x7f5370004d00 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537000ba60 con 0x7f5370003f20 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537000bc20 con 0x7f5384160e30 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f538a466640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5370004d00 0x7f5370005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.006+0000 7f5389c65640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 0x7f53700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f5389464640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370003f20 0x7f53700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f538a466640 1 -- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 msgr2=0x7f53700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f538a466640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5384160e30 0x7f53700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f538a466640 1 -- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370003f20 msgr2=0x7f53700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f538a466640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370003f20 0x7f53700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f538a466640 1 -- 172.21.15.111:0/1595413712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5370003520 con 0x7f5370004d00 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.007+0000 7f538a466640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5370004d00 0x7f5370005180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f537c00a830 tx=0x7f53700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab643651d2ddfe6 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0072d0 con 0x7f5370004d00 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c02f070 con 0x7f5370004d00 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f5388c63640 1 -- 172.21.15.111:0/1595413712 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c007690 con 0x7f5370004d00 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538bec9640 1 -- 172.21.15.111:0/1595413712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5370004d00 msgr2=0x7f5370005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538bec9640 1 --2- 172.21.15.111:0/1595413712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5370004d00 0x7f5370005180 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f537c00a830 tx=0x7f53700057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538a466640 1 -- 172.21.15.111:0/1595413712 reap_dead start 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538bec9640 1 -- 172.21.15.111:0/1595413712 shutdown_connections 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538bec9640 1 -- 172.21.15.111:0/1595413712 >> 172.21.15.111:0/1595413712 conn(0x7f53840b1de0 msgr2=0x7f53840b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538bec9640 1 -- 172.21.15.111:0/1595413712 shutdown_connections 2023-12-04T05:39:27.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.008+0000 7f538bec9640 1 -- 172.21.15.111:0/1595413712 wait complete. 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.009+0000 7f538bec9640 1 Processor -- start 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.009+0000 7f538bec9640 1 -- start start 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f538bec9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370004d00 0x7f53840aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f538bec9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 0x7f538415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f538bec9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f538415ac40 0x7f538413d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f538bec9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538415b0c0 con 0x7f5384152610 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f538bec9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53840af480 con 0x7f5370004d00 2023-12-04T05:39:27.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f538bec9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538412f710 con 0x7f538415ac40 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 0x7f538415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 0x7f538415a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59214/0 (socket says 172.21.15.111:59214) 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 -- 172.21.15.111:0/3437899484 learned_addr learned my addr 172.21.15.111:0/3437899484 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 -- 172.21.15.111:0/3437899484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f538415ac40 msgr2=0x7f538413d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f538415ac40 0x7f538413d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 -- 172.21.15.111:0/3437899484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370004d00 msgr2=0x7f53840aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370004d00 0x7f53840aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 -- 172.21.15.111:0/3437899484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538412eb00 con 0x7f5384152610 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.010+0000 7f5389464640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 0x7f538415a700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5374011970 tx=0x7f5374011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.011+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400fa10 con 0x7f5384152610 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.011+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537400fb70 con 0x7f5384152610 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.011+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538412ede0 con 0x7f5384152610 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.011+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374014440 con 0x7f5384152610 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.011+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538412f3a0 con 0x7f5384152610 2023-12-04T05:39:27.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.012+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5384069360 con 0x7f5384152610 2023-12-04T05:39:27.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.012+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f53740145a0 con 0x7f5384152610 2023-12-04T05:39:27.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.013+0000 7f537a7fc640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f536806b2a0 0x7f536806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.013+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f537408a3b0 con 0x7f5384152610 2023-12-04T05:39:27.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.013+0000 7f5389c65640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f536806b2a0 0x7f536806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.013+0000 7f5389c65640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f536806b2a0 0x7f536806d760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5380001750 tx=0x7f5380009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.017+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5374055220 con 0x7f5384152610 2023-12-04T05:39:27.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.183+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f538415bfa0 con 0x7f5384152610 2023-12-04T05:39:27.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.184+0000 7f537a7fc640 1 -- 172.21.15.111:0/3437899484 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5374058840 con 0x7f5384152610 2023-12-04T05:39:27.185 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:27.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f536806b2a0 msgr2=0x7f536806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f536806b2a0 0x7f536806d760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5380001750 tx=0x7f5380009040 comp rx=0 tx=0).stop 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 msgr2=0x7f538415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 0x7f538415a700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5374011970 tx=0x7f5374011e30 comp rx=0 tx=0).stop 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 shutdown_connections 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f536806b2a0 0x7f536806d760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f538415ac40 0x7f538413d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5384152610 0x7f538415a700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 --2- 172.21.15.111:0/3437899484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5370004d00 0x7f53840aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 >> 172.21.15.111:0/3437899484 conn(0x7f53840b1de0 msgr2=0x7f538415cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 shutdown_connections 2023-12-04T05:39:27.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.186+0000 7f538bec9640 1 -- 172.21.15.111:0/3437899484 wait complete. 2023-12-04T05:39:27.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.301+0000 7f23f12cf640 1 Processor -- start 2023-12-04T05:39:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.301+0000 7f23f12cf640 1 -- start start 2023-12-04T05:39:27.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23f12cf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23ec0b3030 0x7f23ec0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23f12cf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23ec160e30 0x7f23ec161280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23f12cf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0b4ba0 con 0x7f23ec140300 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23f12cf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ec0b4e20 con 0x7f23ec160e30 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23f12cf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ec1406e0 con 0x7f23ec0b3030 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23ec160e30 0x7f23ec161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eaffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f23ec140300 0x7f23ec0b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58858/0 (socket says 172.21.15.111:58858) 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eaffd640 1 -- 172.21.15.111:0/1851950332 learned_addr learned my addr 172.21.15.111:0/1851950332 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23e3fff640 1 --2- 172.21.15.111:0/1851950332 >> v2:172.21.15.192:3300/0 conn(0x7f23ec0b3030 0x7f23ec0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eb7fe640 1 -- 172.21.15.111:0/1851950332 >> v2:172.21.15.192:3300/0 conn(0x7f23ec0b3030 msgr2=0x7f23ec0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eb7fe640 1 --2- 172.21.15.111:0/1851950332 >> v2:172.21.15.192:3300/0 conn(0x7f23ec0b3030 0x7f23ec0b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eb7fe640 1 -- 172.21.15.111:0/1851950332 >> v1:172.21.15.111:6789/0 conn(0x7f23ec140300 legacy=0x7f23ec0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23eb7fe640 1 -- 172.21.15.111:0/1851950332 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec15d7f0 con 0x7f23ec160e30 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.302+0000 7f23ea7fc640 1 -- 172.21.15.111:0/1851950332 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563843806 0 0) 0x7f23ec0b4ba0 con 0x7f23ec140300 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.303+0000 7f23eb7fe640 1 --2- 172.21.15.111:0/1851950332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec161280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0d0ef0 tx=0x7f23e402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3900c379de40ad09 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.303+0000 7f23ea7fc640 1 -- 172.21.15.111:0/1851950332 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e402cbf0 con 0x7f23ec160e30 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.303+0000 7f23ea7fc640 1 -- 172.21.15.111:0/1851950332 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e402cd50 con 0x7f23ec160e30 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.303+0000 7f23ea7fc640 1 -- 172.21.15.111:0/1851950332 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4034660 con 0x7f23ec160e30 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.303+0000 7f23f12cf640 1 -- 172.21.15.111:0/1851950332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 msgr2=0x7f23ec161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.303+0000 7f23f12cf640 1 --2- 172.21.15.111:0/1851950332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec161280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0d0ef0 tx=0x7f23e402be40 comp rx=0 tx=0).stop 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.304+0000 7f23f12cf640 1 -- 172.21.15.111:0/1851950332 shutdown_connections 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.304+0000 7f23f12cf640 1 --2- 172.21.15.111:0/1851950332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec161280 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.304+0000 7f23f12cf640 1 --2- 172.21.15.111:0/1851950332 >> v2:172.21.15.192:3300/0 conn(0x7f23ec0b3030 0x7f23ec0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.304+0000 7f23f12cf640 1 -- 172.21.15.111:0/1851950332 >> 172.21.15.111:0/1851950332 conn(0x7f23ec06a8b0 msgr2=0x7f23ec06acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.304+0000 7f23f12cf640 1 -- 172.21.15.111:0/1851950332 shutdown_connections 2023-12-04T05:39:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.304+0000 7f23f12cf640 1 -- 172.21.15.111:0/1851950332 wait complete. 2023-12-04T05:39:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 Processor -- start 2023-12-04T05:39:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 -- start start 2023-12-04T05:39:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec0b3030 0x7f23ec0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23ec140300 0x7f23ec06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec1f2820 con 0x7f23ec160e30 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0cdfe0 con 0x7f23ec0b3030 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.305+0000 7f23f12cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec1f2a00 con 0x7f23ec140300 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec06cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59228/0 (socket says 172.21.15.111:59228) 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 -- 172.21.15.111:0/4270056946 learned_addr learned my addr 172.21.15.111:0/4270056946 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eaffd640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec0b3030 0x7f23ec0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23e3fff640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23ec140300 0x7f23ec06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 -- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23ec140300 msgr2=0x7f23ec06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23ec140300 0x7f23ec06c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 -- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec0b3030 msgr2=0x7f23ec0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec0b3030 0x7f23ec0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 -- 172.21.15.111:0/4270056946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4009a70 con 0x7f23ec160e30 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eaffd640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec0b3030 0x7f23ec0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23eb7fe640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec06cb60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009e80 tx=0x7f23e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4040650 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e40407d0 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec1f2ce0 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4040b80 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.306+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec0cdcc0 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.307+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec1f3200 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.307+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f23e4040ce0 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.308+0000 7f23e37fe640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23cc06b2a0 0x7f23cc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.308+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23e40ac9b0 con 0x7f23ec160e30 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.308+0000 7f23eaffd640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23cc06b2a0 0x7f23cc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.308+0000 7f23eaffd640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23cc06b2a0 0x7f23cc06d760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000f60 tx=0x7f23dc007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.312+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23ec1f3200 con 0x7f23ec160e30 2023-12-04T05:39:27.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.477+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23ec1f3200 con 0x7f23ec160e30 2023-12-04T05:39:27.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.478+0000 7f23e37fe640 1 -- 172.21.15.111:0/4270056946 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23ec1f3200 con 0x7f23ec160e30 2023-12-04T05:39:27.479 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:27.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23cc06b2a0 msgr2=0x7f23cc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23cc06b2a0 0x7f23cc06d760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000f60 tx=0x7f23dc007c10 comp rx=0 tx=0).stop 2023-12-04T05:39:27.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 msgr2=0x7f23ec06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec06cb60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009e80 tx=0x7f23e4004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 shutdown_connections 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23cc06b2a0 0x7f23cc06d760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ec160e30 0x7f23ec06cb60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23ec140300 0x7f23ec06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 --2- 172.21.15.111:0/4270056946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec0b3030 0x7f23ec0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 >> 172.21.15.111:0/4270056946 conn(0x7f23ec06a8b0 msgr2=0x7f23ec15e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 shutdown_connections 2023-12-04T05:39:27.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.480+0000 7f23f12cf640 1 -- 172.21.15.111:0/4270056946 wait complete. 2023-12-04T05:39:27.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:27.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.592+0000 7f401c6e0640 1 Processor -- start 2023-12-04T05:39:27.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.592+0000 7f401c6e0640 1 -- start start 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.593+0000 7f401c6e0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f40140ce830 0x7f40140cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.593+0000 7f401c6e0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4014070380 0x7f4014070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.593+0000 7f401c6e0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40140cd560 con 0x7f4014070c10 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.593+0000 7f401c6e0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4014159c20 con 0x7f40140ce830 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.593+0000 7f401c6e0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40140cd790 con 0x7f4014070380 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f4019c7b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f40140ce830 0x7f40140cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f401a47c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4014070c10 0x7f40140b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58868/0 (socket says 172.21.15.111:58868) 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f401a47c640 1 -- 172.21.15.111:0/3072667068 learned_addr learned my addr 172.21.15.111:0/3072667068 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:27.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f401ac7d640 1 --2- 172.21.15.111:0/3072667068 >> v2:172.21.15.192:3300/0 conn(0x7f4014070380 0x7f4014070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f4019c7b640 1 -- 172.21.15.111:0/3072667068 >> v2:172.21.15.192:3300/0 conn(0x7f4014070380 msgr2=0x7f4014070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:27.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f4019c7b640 1 --2- 172.21.15.111:0/3072667068 >> v2:172.21.15.192:3300/0 conn(0x7f4014070380 0x7f4014070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f4019c7b640 1 -- 172.21.15.111:0/3072667068 >> v1:172.21.15.111:6789/0 conn(0x7f4014070c10 legacy=0x7f40140b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f4019c7b640 1 -- 172.21.15.111:0/3072667068 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40140cda70 con 0x7f40140ce830 2023-12-04T05:39:27.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f401947a640 1 -- 172.21.15.111:0/3072667068 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158903378 0 0) 0x7f40140cd560 con 0x7f4014070c10 2023-12-04T05:39:27.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.594+0000 7f401ac7d640 1 --2- 172.21.15.111:0/3072667068 >> v2:172.21.15.192:3300/0 conn(0x7f4014070380 0x7f4014070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f4019c7b640 1 --2- 172.21.15.111:0/3072667068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40140ce830 0x7f40140cd020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4014072750 tx=0x7f400402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b7ca8c1fc989b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401947a640 1 -- 172.21.15.111:0/3072667068 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004031830 con 0x7f40140ce830 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401947a640 1 -- 172.21.15.111:0/3072667068 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4004031990 con 0x7f40140ce830 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401947a640 1 -- 172.21.15.111:0/3072667068 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004031d20 con 0x7f40140ce830 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 -- 172.21.15.111:0/3072667068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40140ce830 msgr2=0x7f40140cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 --2- 172.21.15.111:0/3072667068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40140ce830 0x7f40140cd020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4014072750 tx=0x7f400402b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 -- 172.21.15.111:0/3072667068 shutdown_connections 2023-12-04T05:39:27.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 --2- 172.21.15.111:0/3072667068 >> v2:172.21.15.192:3300/0 conn(0x7f4014070380 0x7f4014070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 --2- 172.21.15.111:0/3072667068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40140ce830 0x7f40140cd020 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 -- 172.21.15.111:0/3072667068 >> 172.21.15.111:0/3072667068 conn(0x7f40140b0cf0 msgr2=0x7f40140b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 -- 172.21.15.111:0/3072667068 shutdown_connections 2023-12-04T05:39:27.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.595+0000 7f401c6e0640 1 -- 172.21.15.111:0/3072667068 wait complete. 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.596+0000 7f401c6e0640 1 Processor -- start 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.596+0000 7f401c6e0640 1 -- start start 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401c6e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 0x7f4014134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401c6e0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4014070c10 0x7f4014154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401c6e0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40140ce830 0x7f401415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401c6e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4014153e80 con 0x7f4014070380 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401c6e0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4014129cc0 con 0x7f4014070c10 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401c6e0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4014129700 con 0x7f40140ce830 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401a47c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 0x7f4014134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401a47c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 0x7f4014134340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59248/0 (socket says 172.21.15.111:59248) 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401a47c640 1 -- 172.21.15.111:0/817281032 learned_addr learned my addr 172.21.15.111:0/817281032 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f4019c7b640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4014070c10 0x7f4014154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401ac7d640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40140ce830 0x7f401415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.597+0000 7f401a47c640 1 -- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40140ce830 msgr2=0x7f401415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401a47c640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40140ce830 0x7f401415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401a47c640 1 -- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4014070c10 msgr2=0x7f4014154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401a47c640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4014070c10 0x7f4014154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401a47c640 1 -- 172.21.15.111:0/817281032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009400 con 0x7f4014070380 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401a47c640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 0x7f4014134340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f401000c890 tx=0x7f401000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000a640 con 0x7f4014070380 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401000a7c0 con 0x7f4014070380 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40141299e0 con 0x7f4014070380 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000ab70 con 0x7f4014070380 2023-12-04T05:39:27.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.598+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4014154110 con 0x7f4014070380 2023-12-04T05:39:27.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.599+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4014129560 con 0x7f4014070380 2023-12-04T05:39:27.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.599+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f401000acd0 con 0x7f4014070380 2023-12-04T05:39:27.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.600+0000 7f400affd640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3ff806b2d0 0x7f3ff806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.600+0000 7f4019c7b640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3ff806b2d0 0x7f3ff806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.600+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4010088f80 con 0x7f4014070380 2023-12-04T05:39:27.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.600+0000 7f4019c7b640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3ff806b2d0 0x7f3ff806d790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4004036000 tx=0x7f400403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.604+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40100bbe90 con 0x7f4014070380 2023-12-04T05:39:27.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.771+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f401415b5a0 con 0x7f4014070380 2023-12-04T05:39:27.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.771+0000 7f400affd640 1 -- 172.21.15.111:0/817281032 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f401415b5a0 con 0x7f4014070380 2023-12-04T05:39:27.773 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:27.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3ff806b2d0 msgr2=0x7f3ff806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3ff806b2d0 0x7f3ff806d790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4004036000 tx=0x7f400403c040 comp rx=0 tx=0).stop 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 msgr2=0x7f4014134340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 0x7f4014134340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f401000c890 tx=0x7f401000cd50 comp rx=0 tx=0).stop 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 shutdown_connections 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3ff806b2d0 0x7f3ff806d790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40140ce830 0x7f401415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4014070c10 0x7f4014154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 --2- 172.21.15.111:0/817281032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4014070380 0x7f4014134340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 >> 172.21.15.111:0/817281032 conn(0x7f40140b0cf0 msgr2=0x7f40140ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 shutdown_connections 2023-12-04T05:39:27.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.773+0000 7f401c6e0640 1 -- 172.21.15.111:0/817281032 wait complete. 2023-12-04T05:39:27.785 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.884+0000 7f300a1dc640 1 Processor -- start 2023-12-04T05:39:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.884+0000 7f300a1dc640 1 -- start start 2023-12-04T05:39:27.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.884+0000 7f300a1dc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f300415c060 0x7f30040cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.884+0000 7f300a1dc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300406bdf0 con 0x7f300406b4d0 2023-12-04T05:39:27.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.884+0000 7f300a1dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040cd240 con 0x7f300406b980 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f300a1dc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30040cd520 con 0x7f300415c060 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f30037fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f300415c060 0x7f30040cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f3002ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f300406b4d0 0x7f3004151890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58874/0 (socket says 172.21.15.111:58874) 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f30037fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f300415c060 0x7f30040cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59260/0 (socket says 172.21.15.111:59260) 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f3002ffd640 1 -- 172.21.15.111:0/1698873882 learned_addr learned my addr 172.21.15.111:0/1698873882 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f30037fe640 1 -- 172.21.15.111:0/1698873882 >> v1:172.21.15.192:6789/0 conn(0x7f300406b980 legacy=0x7f3004151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f30037fe640 1 -- 172.21.15.111:0/1698873882 >> v1:172.21.15.111:6789/0 conn(0x7f300406b4d0 legacy=0x7f3004151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.885+0000 7f30037fe640 1 -- 172.21.15.111:0/1698873882 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040cd800 con 0x7f300415c060 2023-12-04T05:39:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f30037fe640 1 --2- 172.21.15.111:0/1698873882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f30040cc8d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40097a0 tx=0x7f2ff402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7184fee7df18327 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f30027fc640 1 -- 172.21.15.111:0/1698873882 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff402e070 con 0x7f300415c060 2023-12-04T05:39:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f30027fc640 1 -- 172.21.15.111:0/1698873882 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff40315a0 con 0x7f300415c060 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f30027fc640 1 -- 172.21.15.111:0/1698873882 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4031930 con 0x7f300415c060 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 -- 172.21.15.111:0/1698873882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 msgr2=0x7f30040cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 --2- 172.21.15.111:0/1698873882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f30040cc8d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40097a0 tx=0x7f2ff402b840 comp rx=0 tx=0).stop 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 -- 172.21.15.111:0/1698873882 shutdown_connections 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 --2- 172.21.15.111:0/1698873882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f30040cc8d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 -- 172.21.15.111:0/1698873882 >> 172.21.15.111:0/1698873882 conn(0x7f30040b1b50 msgr2=0x7f3004070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 -- 172.21.15.111:0/1698873882 shutdown_connections 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.886+0000 7f300a1dc640 1 -- 172.21.15.111:0/1698873882 wait complete. 2023-12-04T05:39:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.887+0000 7f300a1dc640 1 Processor -- start 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 -- start start 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300406b4d0 0x7f30041e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f300406b980 0x7f3004155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f300414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30041555d0 con 0x7f300415c060 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30041e50e0 con 0x7f300406b4d0 2023-12-04T05:39:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f300a1dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30041e53c0 con 0x7f300406b980 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f3003fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f300414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f3003fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f300414f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59264/0 (socket says 172.21.15.111:59264) 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.888+0000 7f3003fff640 1 -- 172.21.15.111:0/856550383 learned_addr learned my addr 172.21.15.111:0/856550383 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f30037fe640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300406b4d0 0x7f30041e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3002ffd640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f300406b980 0x7f3004155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3003fff640 1 -- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f300406b980 msgr2=0x7f3004155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3003fff640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f300406b980 0x7f3004155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3003fff640 1 -- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300406b4d0 msgr2=0x7f30041e4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3003fff640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300406b4d0 0x7f30041e4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3003fff640 1 -- 172.21.15.111:0/856550383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009400 con 0x7f300415c060 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3002ffd640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f300406b980 0x7f3004155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f3003fff640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f300414f7e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c780 tx=0x7f2ffc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004b60 con 0x7f300415c060 2023-12-04T05:39:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc00ce90 con 0x7f300415c060 2023-12-04T05:39:27.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30041e56a0 con 0x7f300415c060 2023-12-04T05:39:27.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00a850 con 0x7f300415c060 2023-12-04T05:39:27.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.889+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30041e5c60 con 0x7f300415c060 2023-12-04T05:39:27.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.890+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300406ec10 con 0x7f300415c060 2023-12-04T05:39:27.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.890+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f2ffc00a9b0 con 0x7f300415c060 2023-12-04T05:39:27.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.891+0000 7f2febfff640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ff006b2a0 0x7f2ff006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:27.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.891+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ffc089d40 con 0x7f300415c060 2023-12-04T05:39:27.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.891+0000 7f30037fe640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ff006b2a0 0x7f2ff006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:27.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.891+0000 7f30037fe640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ff006b2a0 0x7f2ff006d760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4036000 tx=0x7f2ff403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:27.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:27.895+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ffc054bb0 con 0x7f300415c060 2023-12-04T05:39:28.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.094+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30040693d0 con 0x7f300415c060 2023-12-04T05:39:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.094+0000 7f2febfff640 1 -- 172.21.15.111:0/856550383 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f2ffc0581d0 con 0x7f300415c060 2023-12-04T05:39:28.096 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:28.096 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ff006b2a0 msgr2=0x7f2ff006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ff006b2a0 0x7f2ff006d760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4036000 tx=0x7f2ff403d040 comp rx=0 tx=0).stop 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 msgr2=0x7f300414f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f300414f7e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c780 tx=0x7f2ffc00cc40 comp rx=0 tx=0).stop 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 shutdown_connections 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ff006b2a0 0x7f2ff006d760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f300415c060 0x7f300414f7e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f300406b980 0x7f3004155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 --2- 172.21.15.111:0/856550383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f300406b4d0 0x7f30041e4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 >> 172.21.15.111:0/856550383 conn(0x7f30040b1b50 msgr2=0x7f300406d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:28.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.096+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 shutdown_connections 2023-12-04T05:39:28.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.097+0000 7f300a1dc640 1 -- 172.21.15.111:0/856550383 wait complete. 2023-12-04T05:39:28.108 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:28.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.206+0000 7f5d49e4b640 1 Processor -- start 2023-12-04T05:39:28.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.206+0000 7f5d49e4b640 1 -- start start 2023-12-04T05:39:28.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d49e4b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440d1780 0x7f5d440d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d49e4b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5d440b20a0 0x7f5d440d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d49e4b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5d440b2ad0 0x7f5d440ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d49e4b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d440cdcb0 con 0x7f5d440b20a0 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d49e4b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d440d20a0 con 0x7f5d440b2ad0 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d49e4b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440d39e0 con 0x7f5d440d1780 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.207+0000 7f5d42ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5d440b20a0 0x7f5d440d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5d440b20a0 0x7f5d440d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59280/0 (socket says 172.21.15.111:59280) 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 -- 172.21.15.111:0/3221016482 learned_addr learned my addr 172.21.15.111:0/3221016482 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d437fe640 1 --2- 172.21.15.111:0/3221016482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440d1780 0x7f5d440d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d43fff640 1 --2- 172.21.15.111:0/3221016482 >> v2:172.21.15.192:3300/0 conn(0x7f5d440b2ad0 0x7f5d440ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 -- 172.21.15.111:0/3221016482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440d1780 msgr2=0x7f5d440d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 --2- 172.21.15.111:0/3221016482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440d1780 0x7f5d440d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 -- 172.21.15.111:0/3221016482 >> v2:172.21.15.192:3300/0 conn(0x7f5d440b2ad0 msgr2=0x7f5d440ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 --2- 172.21.15.111:0/3221016482 >> v2:172.21.15.192:3300/0 conn(0x7f5d440b2ad0 0x7f5d440ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.208+0000 7f5d42ffd640 1 -- 172.21.15.111:0/3221016482 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440d3cc0 con 0x7f5d440b20a0 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d42ffd640 1 --2- 172.21.15.111:0/3221016482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440b20a0 0x7f5d440d1b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d44162190 tx=0x7f5d3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584114bed06f1d56 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d427fc640 1 -- 172.21.15.111:0/3221016482 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380317e0 con 0x7f5d440b20a0 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d427fc640 1 -- 172.21.15.111:0/3221016482 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38031940 con 0x7f5d440b20a0 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d427fc640 1 -- 172.21.15.111:0/3221016482 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031cd0 con 0x7f5d440b20a0 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 -- 172.21.15.111:0/3221016482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440b20a0 msgr2=0x7f5d440d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/3221016482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440b20a0 0x7f5d440d1b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d44162190 tx=0x7f5d3802b880 comp rx=0 tx=0).stop 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 -- 172.21.15.111:0/3221016482 shutdown_connections 2023-12-04T05:39:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/3221016482 >> v2:172.21.15.192:3300/0 conn(0x7f5d440b2ad0 0x7f5d440ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/3221016482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440b20a0 0x7f5d440d1b60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/3221016482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440d1780 0x7f5d440d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 -- 172.21.15.111:0/3221016482 >> 172.21.15.111:0/3221016482 conn(0x7f5d4409c4e0 msgr2=0x7f5d440d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 -- 172.21.15.111:0/3221016482 shutdown_connections 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.209+0000 7f5d49e4b640 1 -- 172.21.15.111:0/3221016482 wait complete. 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.210+0000 7f5d49e4b640 1 Processor -- start 2023-12-04T05:39:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 -- start start 2023-12-04T05:39:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440b20a0 0x7f5d440ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d440b2ad0 0x7f5d44161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 0x7f5d4406c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440ca940 con 0x7f5d440d1780 2023-12-04T05:39:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441494c0 con 0x7f5d440b2ad0 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.211+0000 7f5d49e4b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441497a0 con 0x7f5d440b20a0 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 0x7f5d4406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 0x7f5d4406c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59290/0 (socket says 172.21.15.111:59290) 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 -- 172.21.15.111:0/2884162047 learned_addr learned my addr 172.21.15.111:0/2884162047 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d437fe640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440b20a0 0x7f5d440ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d42ffd640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d440b2ad0 0x7f5d44161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 -- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440b20a0 msgr2=0x7f5d440ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440b20a0 0x7f5d440ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 -- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d440b2ad0 msgr2=0x7f5d44161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d440b2ad0 0x7f5d44161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 -- 172.21.15.111:0/2884162047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380094d0 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d43fff640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 0x7f5d4406c730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00e7f0 tx=0x7f5d3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c009490 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00c730 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00cac0 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d44149a80 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.212+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4414a040 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.213+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4414a230 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.214+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5d3c00cc20 con 0x7f5d440d1780 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.214+0000 7f5d2bfff640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2a0 0x7f5d3006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.214+0000 7f5d437fe640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2a0 0x7f5d3006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.214+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c089bb0 con 0x7f5d440d1780 2023-12-04T05:39:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.214+0000 7f5d437fe640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2a0 0x7f5d3006d760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d340113e0 tx=0x7f5d3400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.218+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d3c054ad0 con 0x7f5d440d1780 2023-12-04T05:39:28.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.387+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d440693d0 con 0x7f5d440d1780 2023-12-04T05:39:28.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.388+0000 7f5d2bfff640 1 -- 172.21.15.111:0/2884162047 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f5d3c021090 con 0x7f5d440d1780 2023-12-04T05:39:28.390 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2a0 msgr2=0x7f5d3006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2a0 0x7f5d3006d760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d340113e0 tx=0x7f5d3400b040 comp rx=0 tx=0).stop 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 msgr2=0x7f5d4406c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 0x7f5d4406c730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00e7f0 tx=0x7f5d3c00ecb0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 shutdown_connections 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2a0 0x7f5d3006d760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d440d1780 0x7f5d4406c730 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d440b2ad0 0x7f5d44161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 --2- 172.21.15.111:0/2884162047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d440b20a0 0x7f5d440ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 >> 172.21.15.111:0/2884162047 conn(0x7f5d4409c4e0 msgr2=0x7f5d440ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 shutdown_connections 2023-12-04T05:39:28.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.390+0000 7f5d49e4b640 1 -- 172.21.15.111:0/2884162047 wait complete. 2023-12-04T05:39:28.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:28.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.500+0000 7f33c8b1d640 1 Processor -- start 2023-12-04T05:39:28.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33c8b1d640 1 -- start start 2023-12-04T05:39:28.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33c8b1d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33c40cf7d0 0x7f33c4141090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33c8b1d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c413e400 con 0x7f33c406abd0 2023-12-04T05:39:28.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33c8b1d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c40713c0 con 0x7f33c413e020 2023-12-04T05:39:28.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33c8b1d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c4071610 con 0x7f33c40cf7d0 2023-12-04T05:39:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33c2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33c40cf7d0 0x7f33c4141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33bbfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f33c406abd0 0x7f33c4140980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58880/0 (socket says 172.21.15.111:58880) 2023-12-04T05:39:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.501+0000 7f33bbfff640 1 -- 172.21.15.111:0/1781953633 learned_addr learned my addr 172.21.15.111:0/1781953633 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.502+0000 7f33c2d9d640 1 -- 172.21.15.111:0/1781953633 >> v1:172.21.15.192:6789/0 conn(0x7f33c413e020 legacy=0x7f33c406a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.502+0000 7f33c2d9d640 1 -- 172.21.15.111:0/1781953633 >> v1:172.21.15.111:6789/0 conn(0x7f33c406abd0 legacy=0x7f33c4140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.502+0000 7f33c2d9d640 1 -- 172.21.15.111:0/1781953633 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c40718f0 con 0x7f33c40cf7d0 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.502+0000 7f33c2d9d640 1 --2- 172.21.15.111:0/1781953633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c40cf7d0 0x7f33c4141090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009d90 tx=0x7f33bc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135641f63ab08a3e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c1d9b640 1 -- 172.21.15.111:0/1781953633 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc02cbd0 con 0x7f33c40cf7d0 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c1d9b640 1 -- 172.21.15.111:0/1781953633 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc02cd30 con 0x7f33c40cf7d0 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c1d9b640 1 -- 172.21.15.111:0/1781953633 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc02cea0 con 0x7f33c40cf7d0 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1781953633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c40cf7d0 msgr2=0x7f33c4141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1781953633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c40cf7d0 0x7f33c4141090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009d90 tx=0x7f33bc02be20 comp rx=0 tx=0).stop 2023-12-04T05:39:28.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1781953633 shutdown_connections 2023-12-04T05:39:28.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1781953633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c40cf7d0 0x7f33c4141090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1781953633 >> 172.21.15.111:0/1781953633 conn(0x7f33c40b1c00 msgr2=0x7f33c40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:28.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1781953633 shutdown_connections 2023-12-04T05:39:28.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.503+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1781953633 wait complete. 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.504+0000 7f33c8b1d640 1 Processor -- start 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 -- start start 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 0x7f33c41395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c40cf7d0 0x7f33c4140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33c413e020 0x7f33c4138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41544f0 con 0x7f33c406abd0 2023-12-04T05:39:28.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4152ff0 con 0x7f33c413e020 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c8b1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41547b0 con 0x7f33c40cf7d0 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 0x7f33c41395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 0x7f33c41395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59310/0 (socket says 172.21.15.111:59310) 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c259c640 1 -- 172.21.15.111:0/1000605106 learned_addr learned my addr 172.21.15.111:0/1000605106 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33c2d9d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33c413e020 0x7f33c4138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.505+0000 7f33bbfff640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c40cf7d0 0x7f33c4140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c259c640 1 -- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c40cf7d0 msgr2=0x7f33c4140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c259c640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c40cf7d0 0x7f33c4140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c259c640 1 -- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33c413e020 msgr2=0x7f33c4138560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c259c640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33c413e020 0x7f33c4138560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c259c640 1 -- 172.21.15.111:0/1000605106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc009a70 con 0x7f33c406abd0 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33bbfff640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c40cf7d0 0x7f33c4140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:28.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c259c640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 0x7f33c41395d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f33b400b420 tx=0x7f33b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:28.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4011830 con 0x7f33c406abd0 2023-12-04T05:39:28.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b4011990 con 0x7f33c406abd0 2023-12-04T05:39:28.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c41524d0 con 0x7f33c406abd0 2023-12-04T05:39:28.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4011d40 con 0x7f33c406abd0 2023-12-04T05:39:28.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.506+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c4152a30 con 0x7f33c406abd0 2023-12-04T05:39:28.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.507+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c4067460 con 0x7f33c406abd0 2023-12-04T05:39:28.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.507+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f33b401b460 con 0x7f33c406abd0 2023-12-04T05:39:28.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.508+0000 7f33baffd640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f33a406b2d0 0x7f33a406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:28.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.508+0000 7f33bbfff640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f33a406b2d0 0x7f33a406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:28.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.508+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33b4089750 con 0x7f33c406abd0 2023-12-04T05:39:28.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.508+0000 7f33bbfff640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f33a406b2d0 0x7f33a406d790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33c41f41d0 tx=0x7f33ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:28.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.512+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33b40545c0 con 0x7f33c406abd0 2023-12-04T05:39:28.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.715+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c4138340 con 0x7f33c406abd0 2023-12-04T05:39:28.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.716+0000 7f33baffd640 1 -- 172.21.15.111:0/1000605106 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f33b4057be0 con 0x7f33c406abd0 2023-12-04T05:39:28.717 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:28.717 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:28.718 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:28.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f33a406b2d0 msgr2=0x7f33a406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f33a406b2d0 0x7f33a406d790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33c41f41d0 tx=0x7f33ac009040 comp rx=0 tx=0).stop 2023-12-04T05:39:28.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 msgr2=0x7f33c41395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:28.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 0x7f33c41395d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f33b400b420 tx=0x7f33b400b8e0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 shutdown_connections 2023-12-04T05:39:28.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f33a406b2d0 0x7f33a406d790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33c413e020 0x7f33c4138560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c40cf7d0 0x7f33c4140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.717+0000 7f33c8b1d640 1 --2- 172.21.15.111:0/1000605106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33c406abd0 0x7f33c41395d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:28.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.718+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 >> 172.21.15.111:0/1000605106 conn(0x7f33c40b1c00 msgr2=0x7f33c40cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:28.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.718+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 shutdown_connections 2023-12-04T05:39:28.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:28.718+0000 7f33c8b1d640 1 -- 172.21.15.111:0/1000605106 wait complete. 2023-12-04T05:39:29.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-04T05:39:16.542901+0000', 'modified': '2023-12-04T05:39:18.555593+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6836/4088659597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 4088659597}, {'type': 'v1', 'addr': '172.21.15.111:6836', 'nonce': 4088659597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-04T05:39:29.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-04T05:39:29.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-04T05:39:29.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.233+0000 7f49d59be640 1 Processor -- start 2023-12-04T05:39:29.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.233+0000 7f49d59be640 1 -- start start 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49d59be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49d00b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49d59be640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f49d00b52e0 0x7f49d00b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49d59be640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d00d1650 con 0x7f49d00d11b0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49d59be640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d0160d30 con 0x7f49d00b52e0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49d59be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00d1930 con 0x7f49d0160950 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49ce7fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f49d00d11b0 0x7f49d00b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58896/0 (socket says 172.21.15.111:58896) 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49ce7fc640 1 -- 172.21.15.111:0/3721322708 learned_addr learned my addr 172.21.15.111:0/3721322708 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cf7fe640 1 --2- 172.21.15.111:0/3721322708 >> v2:172.21.15.192:3300/0 conn(0x7f49d00b52e0 0x7f49d00b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49d00b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331842215 0 0) 0x7f49d00d1650 con 0x7f49d00d11b0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b8003520 con 0x7f49d00d11b0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809617406 0 0) 0x7f49b8003520 con 0x7f49d00d11b0 2023-12-04T05:39:29.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 msgr2=0x7f49d00b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49d00b8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 >> v2:172.21.15.192:3300/0 conn(0x7f49d00b52e0 msgr2=0x7f49d00b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 --2- 172.21.15.111:0/3721322708 >> v2:172.21.15.192:3300/0 conn(0x7f49d00b52e0 0x7f49d00b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00d1c10 con 0x7f49d00d11b0 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f49c4001670 con 0x7f49d00d11b0 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49d00b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 >> v1:172.21.15.111:6789/0 conn(0x7f49d00d11b0 legacy=0x7f49d00b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.234+0000 7f49cdffb640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cdffb640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b80047c0 0x7f49b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cdffb640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0160d30 con 0x7f49b80047c0 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00d1930 con 0x7f49b8003970 2023-12-04T05:39:29.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b800b980 con 0x7f49d0160950 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cf7fe640 1 --2- 172.21.15.111:0/3721322708 >> v2:172.21.15.192:3300/0 conn(0x7f49d00b52e0 0x7f49d00b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b80047c0 0x7f49b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49cf7fe640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 -- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 msgr2=0x7f49b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49d0160950 0x7f49b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 -- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b8003970 msgr2=0x7f49b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 -- 172.21.15.111:0/3721322708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c0009400 con 0x7f49b80047c0 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.235+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b80047c0 0x7f49b8004c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c240 tx=0x7f49b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cb3de28b6e296d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c402cb00 con 0x7f49b80047c0 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c402cc60 con 0x7f49b80047c0 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49cdffb640 1 -- 172.21.15.111:0/3721322708 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4005af0 con 0x7f49b80047c0 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49d59be640 1 -- 172.21.15.111:0/3721322708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b80047c0 msgr2=0x7f49b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49d59be640 1 --2- 172.21.15.111:0/3721322708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b80047c0 0x7f49b8004c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c240 tx=0x7f49b80057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49cf7fe640 1 -- 172.21.15.111:0/3721322708 reap_dead start 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49d59be640 1 -- 172.21.15.111:0/3721322708 shutdown_connections 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49d59be640 1 -- 172.21.15.111:0/3721322708 >> 172.21.15.111:0/3721322708 conn(0x7f49d0072220 msgr2=0x7f49d00cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49d59be640 1 -- 172.21.15.111:0/3721322708 shutdown_connections 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.236+0000 7f49d59be640 1 -- 172.21.15.111:0/3721322708 wait complete. 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.237+0000 7f49d59be640 1 Processor -- start 2023-12-04T05:39:29.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.237+0000 7f49d59be640 1 -- start start 2023-12-04T05:39:29.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49d59be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b80047c0 0x7f49d01f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49d59be640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 0x7f49d015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49d59be640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49d015ee80 0x7f49d015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49d59be640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015ad70 con 0x7f49d015ea00 2023-12-04T05:39:29.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49d59be640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00c7f70 con 0x7f49d015ee80 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49d59be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00c81c0 con 0x7f49b80047c0 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49ce7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 0x7f49d015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49ce7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 0x7f49d015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59328/0 (socket says 172.21.15.111:59328) 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49ce7fc640 1 -- 172.21.15.111:0/1605485122 learned_addr learned my addr 172.21.15.111:0/1605485122 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.238+0000 7f49ce7fc640 1 -- 172.21.15.111:0/1605485122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b80047c0 msgr2=0x7f49d01f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b80047c0 0x7f49d01f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49ce7fc640 1 -- 172.21.15.111:0/1605485122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49d015ee80 msgr2=0x7f49d015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49d015ee80 0x7f49d015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49ce7fc640 1 -- 172.21.15.111:0/1605485122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00d2870 con 0x7f49d015ea00 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 0x7f49d015a2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c240 tx=0x7f49c4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49b7fff640 1 -- 172.21.15.111:0/1605485122 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4039070 con 0x7f49d015ea00 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49b7fff640 1 -- 172.21.15.111:0/1605485122 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4004bf0 con 0x7f49d015ea00 2023-12-04T05:39:29.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49b7fff640 1 -- 172.21.15.111:0/1605485122 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c403c040 con 0x7f49d015ea00 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d00c84a0 con 0x7f49d015ea00 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.239+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d00c8a60 con 0x7f49d015ea00 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.240+0000 7f49b7fff640 1 -- 172.21.15.111:0/1605485122 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f49c4032070 con 0x7f49d015ea00 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.241+0000 7f49b7fff640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f49bc06b2a0 0x7f49bc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.241+0000 7f49b7fff640 1 -- 172.21.15.111:0/1605485122 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49c40ab040 con 0x7f49d015ea00 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.241+0000 7f49ceffd640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f49bc06b2a0 0x7f49bc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.241+0000 7f49ceffd640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f49bc06b2a0 0x7f49bc06d760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49c00098e0 tx=0x7f49c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.242+0000 7f49d862c740 1 Processor -- start 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.242+0000 7f49d862c740 1 -- start start 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49d862c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2a9480 0x55633e2d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49d862c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e269d10 con 0x55633e2d5fa0 2023-12-04T05:39:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e21bb90 con 0x55633e2d6380 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55633e219280 con 0x55633e2a9480 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2a8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59330/0 (socket says 172.21.15.111:59330) 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 -- 172.21.15.111:0/585383966 learned_addr learned my addr 172.21.15.111:0/585383966 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 -- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2a9480 msgr2=0x55633e2d9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2a9480 0x55633e2d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 -- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 msgr2=0x55633e2d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2d8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 -- 172.21.15.111:0/585383966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55633e1da500 con 0x55633e2d5fa0 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49ceffd640 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2a8f40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011c70 tx=0x7f49c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bb92630e235fc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49b57fa640 1 -- 172.21.15.111:0/585383966 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0015dd0 con 0x55633e2d5fa0 2023-12-04T05:39:29.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49b57fa640 1 -- 172.21.15.111:0/585383966 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0011ca0 con 0x55633e2d5fa0 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.243+0000 7f49b57fa640 1 -- 172.21.15.111:0/585383966 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0021440 con 0x55633e2d5fa0 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 -- 172.21.15.111:0/585383966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 msgr2=0x55633e2a8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2a8f40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011c70 tx=0x7f49c0014000 comp rx=0 tx=0).stop 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 -- 172.21.15.111:0/585383966 shutdown_connections 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2a9480 0x55633e2d9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 --2- 172.21.15.111:0/585383966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2a8f40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 -- 172.21.15.111:0/585383966 >> 172.21.15.111:0/585383966 conn(0x55633e2ab1b0 msgr2=0x55633e2f3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 -- 172.21.15.111:0/585383966 shutdown_connections 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.244+0000 7f49d862c740 1 -- 172.21.15.111:0/585383966 wait complete. 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 -- start start 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 Processor -- start 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2ee360 0x55633e307500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e214b70 con 0x55633e2d5fa0 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e1cee00 con 0x55633e2d6380 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55633e1ceaa0 con 0x55633e2ee360 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49d862c740 10 client.0 _ll_register_callbacks cb 0x55633e1d32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59346/0 (socket says 172.21.15.111:59346) 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.245+0000 7f49ceffd640 1 -- 172.21.15.111:0/937996120 learned_addr learned my addr 172.21.15.111:0/937996120 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49cf7fe640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2ee360 0x55633e307500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ceffd640 1 -- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2ee360 msgr2=0x55633e307500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ceffd640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2ee360 0x55633e307500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ceffd640 1 -- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 msgr2=0x55633e2c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ceffd640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ceffd640 1 -- 172.21.15.111:0/937996120 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c0011920 con 0x55633e2d5fa0 2023-12-04T05:39:29.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49cf7fe640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2ee360 0x55633e307500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ceffd640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2c38b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49c0008010 tx=0x7f49c0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29dc381fca961b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ae7fc640 1 -- 172.21.15.111:0/937996120 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0011e10 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ae7fc640 1 -- 172.21.15.111:0/937996120 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49c0031070 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55633e20db50 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ae7fc640 1 -- 172.21.15.111:0/937996120 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c0021b50 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 10 client.4701 fetch_fsmap learned FSMap version 6 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55633e2131f0 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ae7fc640 1 -- 172.21.15.111:0/937996120 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55633e20db50 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49ae7fc640 1 -- 172.21.15.111:0/937996120 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f49c0021cb0 con 0x55633e2d5fa0 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 10 client.4701 fetch_fsmap finished waiting for FSMap version 6 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 10 client.4701 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-04T05:39:29.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 20 client.4701 populate_metadata read hostname 'smithi111' 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2c6800 0x55633e2c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 4 client.4701 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.246+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 --> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55633e20d870 con 0x55633e2c6800 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2c6800 0x55633e2c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49ae7fc640 10 client.4701 ms_handle_connect on v2:172.21.15.111:6834/4088659597 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2c6800 0x55633e2c6be0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db33da8ee5e0007 server_cookie=2824fad4070ba9c2 in_seq=0 out_seq=0 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49ae7fc640 1 -- 172.21.15.111:0/937996120 <== mds.0 v2:172.21.15.111:6834/4088659597 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55633e20d870 con 0x55633e2c6800 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49ae7fc640 10 client.4701 handle_command_reply: tid=0 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 client.4701 shutdown 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 msgr2=0x55633e2c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2c38b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49c0008010 tx=0x7f49c0009190 comp rx=0 tx=0).stop 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 shutdown_connections 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2c6800 0x55633e2c6be0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2ee360 0x55633e307500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49ae7fc640 0 client.4701 ms_handle_reset on v2:172.21.15.111:6834/4088659597 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d6380 0x55633e2c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 --2- 172.21.15.111:0/937996120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2c38b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.247+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 >> 172.21.15.111:0/937996120 conn(0x55633e2a9480 msgr2=0x55633e2acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.248+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 shutdown_connections 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.248+0000 7f49d862c740 1 -- 172.21.15.111:0/937996120 wait complete. 2023-12-04T05:39:29.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.248+0000 7f49d862c740 20 client.4701 trim_cache size 0 max 16384 2023-12-04T05:39:29.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.263+0000 7f49d862c740 1 Processor -- start 2023-12-04T05:39:29.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.263+0000 7f49d862c740 1 -- start start 2023-12-04T05:39:29.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2c6800 0x55633e2d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49d862c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 0x55633e2eef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49d862c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e194270 con 0x55633e2d5fa0 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e2dbaf0 con 0x55633e2c6800 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55633e2daf40 con 0x55633e2d6380 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49ce7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49ce7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59360/0 (socket says 172.21.15.111:59360) 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.264+0000 7f49ce7fc640 1 -- 172.21.15.111:0/489757701 learned_addr learned my addr 172.21.15.111:0/489757701 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49ce7fc640 1 -- 172.21.15.111:0/489757701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 msgr2=0x55633e2eef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 0x55633e2eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49ce7fc640 1 -- 172.21.15.111:0/489757701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2c6800 msgr2=0x55633e2d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2c6800 0x55633e2d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49ce7fc640 1 -- 172.21.15.111:0/489757701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55633e1b0d00 con 0x55633e2d5fa0 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2aebb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49c40799e0 tx=0x7f49c404c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8898cbb1af0970 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49b5ffb640 1 -- 172.21.15.111:0/489757701 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c407de30 con 0x55633e2d5fa0 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49b5ffb640 1 -- 172.21.15.111:0/489757701 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4081fe0 con 0x55633e2d5fa0 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49b5ffb640 1 -- 172.21.15.111:0/489757701 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4007d10 con 0x55633e2d5fa0 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49d862c740 1 -- 172.21.15.111:0/489757701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 msgr2=0x55633e2aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.265+0000 7f49d862c740 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2aebb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49c40799e0 tx=0x7f49c404c040 comp rx=0 tx=0).stop 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 -- 172.21.15.111:0/489757701 shutdown_connections 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 0x55633e2eef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2d5fa0 0x55633e2aebb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 --2- 172.21.15.111:0/489757701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2c6800 0x55633e2d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 -- 172.21.15.111:0/489757701 >> 172.21.15.111:0/489757701 conn(0x55633e2a9480 msgr2=0x55633e2f6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 -- 172.21.15.111:0/489757701 shutdown_connections 2023-12-04T05:39:29.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 -- 172.21.15.111:0/489757701 wait complete. 2023-12-04T05:39:29.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 -- start start 2023-12-04T05:39:29.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.266+0000 7f49d862c740 1 Processor -- start 2023-12-04T05:39:29.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 0x55633e305c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d5fa0 0x55633e306150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 0x55633e35e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e35f450 con 0x55633e2c6800 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55633e316940 con 0x55633e2d5fa0 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55633e306a80 con 0x55633e2d6380 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49d862c740 10 client.0 _ll_register_callbacks cb 0x7f49c0021860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 0x55633e305c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 0x55633e305c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59362/0 (socket says 172.21.15.111:59362) 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 -- 172.21.15.111:0/3086215545 learned_addr learned my addr 172.21.15.111:0/3086215545 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 -- 172.21.15.111:0/3086215545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 msgr2=0x55633e35e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 0x55633e35e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 -- 172.21.15.111:0/3086215545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d5fa0 msgr2=0x55633e306150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d5fa0 0x55633e306150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 -- 172.21.15.111:0/3086215545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d00c8a60 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ceffd640 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 0x55633e305c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006da0 tx=0x7f49c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788a646da96cfba server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.267+0000 7f49ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ae7fc640 1 -- 172.21.15.111:0/3086215545 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0011e10 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ae7fc640 1 -- 172.21.15.111:0/3086215545 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49c0031070 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49c0011ca0 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ae7fc640 1 -- 172.21.15.111:0/3086215545 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c000d040 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 10 client.4707 fetch_fsmap learned FSMap version 6 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55633e1d64f0 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ae7fc640 1 -- 172.21.15.111:0/3086215545 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0011ca0 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ae7fc640 1 -- 172.21.15.111:0/3086215545 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f49c0036810 con 0x55633e2c6800 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 10 client.4707 fetch_fsmap finished waiting for FSMap version 6 2023-12-04T05:39:29.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 10 client.4707 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 20 client.4707 populate_metadata read hostname 'smithi111' 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2ee360 0x55633e35e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 4 client.4707 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 --> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55633df156c0 con 0x55633e2ee360 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2ee360 0x55633e35e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ae7fc640 10 client.4707 ms_handle_connect on v2:172.21.15.111:6834/4088659597 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.268+0000 7f49ce7fc640 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2ee360 0x55633e35e740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a49a12746c6502 server_cookie=ec7811ba4bcb1275 in_seq=0 out_seq=0 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49ae7fc640 1 -- 172.21.15.111:0/3086215545 <== mds.0 v2:172.21.15.111:6834/4088659597 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55633df156c0 con 0x55633e2ee360 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49ae7fc640 10 client.4707 handle_command_reply: tid=0 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 client.4707 shutdown 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 msgr2=0x55633e305c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 0x55633e305c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006da0 tx=0x7f49c0004730 comp rx=0 tx=0).stop 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 shutdown_connections 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:6834/4088659597,v1:172.21.15.111:6836/4088659597] conn(0x55633e2ee360 0x55633e35e740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55633e2d6380 0x55633e35e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633e2d5fa0 0x55633e306150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 --2- 172.21.15.111:0/3086215545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55633e2c6800 0x55633e305c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49ae7fc640 0 client.4707 ms_handle_reset on v2:172.21.15.111:6834/4088659597 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 >> 172.21.15.111:0/3086215545 conn(0x55633e2a9480 msgr2=0x55633e2f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 shutdown_connections 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 1 -- 172.21.15.111:0/3086215545 wait complete. 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d862c740 20 client.4707 trim_cache size 0 max 16384 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "9686316e-a01b-4981-b0b7-e21bdb185d5e", 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-12-04T05:39:29.277 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4284, 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 12.051118551, 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 6, 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 20, 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 20, 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 19.087721115000001 2023-12-04T05:39:29.278 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f49bc06b2a0 msgr2=0x7f49bc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f49bc06b2a0 0x7f49bc06d760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49c00098e0 tx=0x7f49c0008040 comp rx=0 tx=0).stop 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 msgr2=0x7f49d015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 0x7f49d015a2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c240 tx=0x7f49c4001a00 comp rx=0 tx=0).stop 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 shutdown_connections 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f49bc06b2a0 0x7f49bc06d760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49d015ee80 0x7f49d015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49d015ea00 0x7f49d015a2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 --2- 172.21.15.111:0/1605485122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b80047c0 0x7f49d01f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.269+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 >> 172.21.15.111:0/1605485122 conn(0x7f49d0072220 msgr2=0x7f49d00d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.270+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 shutdown_connections 2023-12-04T05:39:29.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.270+0000 7f49d59be640 1 -- 172.21.15.111:0/1605485122 wait complete. 2023-12-04T05:39:29.281 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-12-04T05:39:29.306 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-12-04T05:39:29.306 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-04T05:39:29.306 INFO:tasks.ceph_fuse:client.0 config is {} 2023-12-04T05:39:29.306 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-04T05:39:29.306 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:29.307 INFO:tasks.ceph_fuse:client.2 config is {} 2023-12-04T05:39:29.307 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-04T05:39:29.307 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:29.307 INFO:tasks.ceph_fuse:client.1 config is {} 2023-12-04T05:39:29.307 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-04T05:39:29.307 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:29.307 INFO:tasks.ceph_fuse:client.3 config is {} 2023-12-04T05:39:29.307 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-04T05:39:29.307 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:29.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.307 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:39:29.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-12-04T05:39:29.391 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2023-12-04T05:39:29.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:39:29.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.393 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-04T05:39:29.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2023-12-04T05:39:29.485 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2023-12-04T05:39:29.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:39:29.487 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-12-04T05:39:29.487 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:39:29.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.585+0000 7fe39b9de640 1 Processor -- start 2023-12-04T05:39:29.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.585+0000 7fe39b9de640 1 -- start start 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39b9de640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe39413cee0 0x7fe39406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39b9de640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe39406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39b9de640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3940aea20 con 0x7fe39415a490 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39b9de640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39413d2c0 con 0x7fe39413cee0 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39b9de640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39415b3a0 con 0x7fe394140200 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe399f7b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe39415a490 0x7fe394071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58900/0 (socket says 172.21.15.111:58900) 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe399f7b640 1 -- 172.21.15.111:0/1965932602 learned_addr learned my addr 172.21.15.111:0/1965932602 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39977a640 1 --2- 172.21.15.111:0/1965932602 >> v2:172.21.15.192:3300/0 conn(0x7fe39413cee0 0x7fe39406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe398f79640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe39406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe38bfff640 1 -- 172.21.15.111:0/1965932602 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142552392 0 0) 0x7fe3940aea20 con 0x7fe39415a490 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe38bfff640 1 -- 172.21.15.111:0/1965932602 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe380003520 con 0x7fe39415a490 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe38bfff640 1 -- 172.21.15.111:0/1965932602 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203240274 0 0) 0x7fe380003520 con 0x7fe39415a490 2023-12-04T05:39:29.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39977a640 1 -- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 msgr2=0x7fe39406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:29.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39977a640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe39406c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39977a640 1 -- 172.21.15.111:0/1965932602 >> v1:172.21.15.111:6789/0 conn(0x7fe39415a490 legacy=0x7fe394071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe39977a640 1 -- 172.21.15.111:0/1965932602 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39415b0c0 con 0x7fe39413cee0 2023-12-04T05:39:29.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.586+0000 7fe398f79640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe39406c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:29.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.587+0000 7fe39977a640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39406c250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3900097a0 tx=0x7fe39002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf078a7f7449bb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.587+0000 7fe38bfff640 1 -- 172.21.15.111:0/1965932602 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c0019f0 con 0x7fe39413cee0 2023-12-04T05:39:29.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.587+0000 7fe38bfff640 1 -- 172.21.15.111:0/1965932602 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39002e050 con 0x7fe39413cee0 2023-12-04T05:39:29.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.587+0000 7fe38bfff640 1 -- 172.21.15.111:0/1965932602 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390030710 con 0x7fe39413cee0 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 -- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 msgr2=0x7fe39406c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39406c250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3900097a0 tx=0x7fe39002b710 comp rx=0 tx=0).stop 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 -- 172.21.15.111:0/1965932602 shutdown_connections 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe39406c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 --2- 172.21.15.111:0/1965932602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39406c250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 -- 172.21.15.111:0/1965932602 >> 172.21.15.111:0/1965932602 conn(0x7fe39406a4c0 msgr2=0x7fe39406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 -- 172.21.15.111:0/1965932602 shutdown_connections 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.588+0000 7fe39b9de640 1 -- 172.21.15.111:0/1965932602 wait complete. 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.589+0000 7fe39b9de640 1 Processor -- start 2023-12-04T05:39:29.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.589+0000 7fe39b9de640 1 -- start start 2023-12-04T05:39:29.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39413e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe394070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 0x7fe3941f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940b26d0 con 0x7fe39415a490 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940ad6a0 con 0x7fe39413cee0 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39413d8a0 con 0x7fe394140200 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 0x7fe3941f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 0x7fe3941f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59378/0 (socket says 172.21.15.111:59378) 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 -- 172.21.15.111:0/3943362986 learned_addr learned my addr 172.21.15.111:0/3943362986 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe398f79640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe394070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39977a640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39413e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 -- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 msgr2=0x7fe394070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe394070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 -- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 msgr2=0x7fe39413e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39413e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 -- 172.21.15.111:0/3943362986 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384009400 con 0x7fe39415a490 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe399f7b640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 0x7fe3941f3900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00cc00 tx=0x7fe38c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00ce10 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38c012400 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe390009400 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.590+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c0127b0 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.591+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39413de60 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.592+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39413d700 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.592+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe38c012910 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.592+0000 7fe389ffb640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe37806b2a0 0x7fe37806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.592+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe38c08a340 con 0x7fe39415a490 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.592+0000 7fe39977a640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe37806b2a0 0x7fe37806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:29.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.593+0000 7fe39977a640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe37806b2a0 0x7fe37806d760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe390034000 tx=0x7fe390008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:29.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.596+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe39413d700 con 0x7fe39415a490 2023-12-04T05:39:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.797+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe39413d700 con 0x7fe39415a490 2023-12-04T05:39:29.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.797+0000 7fe389ffb640 1 -- 172.21.15.111:0/3943362986 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe39413d700 con 0x7fe39415a490 2023-12-04T05:39:29.799 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe37806b2a0 msgr2=0x7fe37806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe37806b2a0 0x7fe37806d760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe390034000 tx=0x7fe390008030 comp rx=0 tx=0).stop 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 msgr2=0x7fe3941f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 0x7fe3941f3900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00cc00 tx=0x7fe38c00a430 comp rx=0 tx=0).stop 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 shutdown_connections 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe37806b2a0 0x7fe37806d760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe39415a490 0x7fe3941f3900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe394140200 0x7fe394070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 --2- 172.21.15.111:0/3943362986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe39413cee0 0x7fe39413e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 >> 172.21.15.111:0/3943362986 conn(0x7fe39406a4c0 msgr2=0x7fe394137f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 shutdown_connections 2023-12-04T05:39:29.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:29.799+0000 7fe39b9de640 1 -- 172.21.15.111:0/3943362986 wait complete. 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:39:29.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:39:29.812 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:39:29.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.812 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-12-04T05:39:29.827 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:29.827 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:29.828 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:29.828 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:29.828 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:29.828 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:29.828 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:29.828 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:29.829 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-12-04T05:39:29.829 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:29.829 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:29.829 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-12-04T05:39:29.829 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:29.829 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:29.830 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:39:29.830 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:29.830 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:39:29.830 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42148sec preferred_lft 42148sec 2023-12-04T05:39:29.830 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-12-04T05:39:29.830 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:29.831 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-04T05:39:29.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.832 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:29.832 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:29.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2023-12-04T05:39:29.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2023-12-04T05:39:29.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2023-12-04T05:39:29.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-04T05:39:29.833 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:29.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:29.995 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:39:30.021 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-12-04T05:39:30.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.023 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-12-04T05:39:30.096 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-12-04T05:39:30.096 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:39:30.096 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:39:30.097 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:39:30.097 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:39:30.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.097 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:30.097 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:30.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:39:30.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:39:30.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:39:30.098 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:30.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.392 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:39:30.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.407 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-12-04T05:39:30.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.462 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:30.462 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:30.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:30.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-04T05:39:30.462 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:30.545 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-04T05:39:30.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-04T05:39:30.545 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:30.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.687 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:30.687 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:30.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-12-04T05:39:30.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2023-12-04T05:39:30.687 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:30.729 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-04T05:39:30.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:30.729 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:30.784 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:39:30.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:30.784 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:30.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:39:30.905 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:30.960 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:39:30.960 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:39:30.960 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:39:30.960 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:30.962 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:30.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:30.963 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:39:31.014 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:39:31.014 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-04T05:39:31.056 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:39:31.084 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:31.139 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:39:31.139 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:39:31.139 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:39:31.139 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:31.140 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:31.141 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:31.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:31.142 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:39:31.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2023-12-04T05:39:31.180+0000 7f3515646480 -1 init, newargv = 0x55dbde0ab350 newargc=15 2023-12-04T05:39:31.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[98049]: starting ceph client 2023-12-04T05:39:31.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[98049]: starting fuse 2023-12-04T05:39:31.196 INFO:teuthology.orchestra.run.smithi111.stdout:41 2023-12-04T05:39:31.196 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-12-04T05:39:31.197 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2023-12-04T05:39:31.198 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-12-04T05:39:31.319 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.98049.asok 2023-12-04T05:39:31.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:39:31.329 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.98049.asok 2023-12-04T05:39:31.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:31.329 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.98049.asok status 2023-12-04T05:39:31.464 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "98049", 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-12-04T05:39:31.465 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4719, 2023-12-04T05:39:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-12-04T05:39:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-12-04T05:39:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-12-04T05:39:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4719 2023-12-04T05:39:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:31.466 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:39:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:39:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:39:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2322123705 2023-12-04T05:39:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:39:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:31.467 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:39:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:39:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:39:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2322123705 2023-12-04T05:39:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4719 192.168.0.1:0/2322123705", 2023-12-04T05:39:31.468 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/2322123705", 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 6, 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 20, 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-12-04T05:39:31.469 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:39:31.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:39:31.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.570+0000 7fa8e3e56640 1 Processor -- start 2023-12-04T05:39:31.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.570+0000 7fa8e3e56640 1 -- start start 2023-12-04T05:39:31.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e3e56640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa8dc06cae0 0x7fa8dc06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:31.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e3e56640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8dc0b3310 0x7fa8dc159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:31.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e3e56640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc154220 con 0x7fa8dc06c250 2023-12-04T05:39:31.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e3e56640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc15d2f0 con 0x7fa8dc06cae0 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e3e56640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc06d400 con 0x7fa8dc0b3310 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e1bf2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa8dc06cae0 0x7fa8dc06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e1bf2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa8dc06cae0 0x7fa8dc06cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43470/0 (socket says 172.21.15.111:43470) 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e23f3640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa8dc06c250 0x7fa8dc15dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51050/0 (socket says 172.21.15.111:51050) 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.571+0000 7fa8e1bf2640 1 -- 172.21.15.111:0/2654960905 learned_addr learned my addr 172.21.15.111:0/2654960905 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.572+0000 7fa8e13f1640 1 --2- 172.21.15.111:0/2654960905 >> v2:172.21.15.192:3300/0 conn(0x7fa8dc0b3310 0x7fa8dc159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.572+0000 7fa8e1bf2640 1 -- 172.21.15.111:0/2654960905 >> v2:172.21.15.192:3300/0 conn(0x7fa8dc0b3310 msgr2=0x7fa8dc159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.572+0000 7fa8e1bf2640 1 --2- 172.21.15.111:0/2654960905 >> v2:172.21.15.192:3300/0 conn(0x7fa8dc0b3310 0x7fa8dc159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.572+0000 7fa8e1bf2640 1 -- 172.21.15.111:0/2654960905 >> v1:172.21.15.111:6789/0 conn(0x7fa8dc06c250 legacy=0x7fa8dc15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:31.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.572+0000 7fa8e1bf2640 1 -- 172.21.15.111:0/2654960905 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc15a5c0 con 0x7fa8dc06cae0 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.572+0000 7fa8e1bf2640 1 --2- 172.21.15.111:0/2654960905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc06cec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0718b0 tx=0x7fa8d802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdf8fde9f40cddc server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e0bf0640 1 -- 172.21.15.111:0/2654960905 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d802cb20 con 0x7fa8dc06cae0 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e0bf0640 1 -- 172.21.15.111:0/2654960905 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d802cc80 con 0x7fa8dc06cae0 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e0bf0640 1 -- 172.21.15.111:0/2654960905 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80343f0 con 0x7fa8dc06cae0 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 -- 172.21.15.111:0/2654960905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc06cae0 msgr2=0x7fa8dc06cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/2654960905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc06cec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0718b0 tx=0x7fa8d802bf80 comp rx=0 tx=0).stop 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 -- 172.21.15.111:0/2654960905 shutdown_connections 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/2654960905 >> v2:172.21.15.192:3300/0 conn(0x7fa8dc0b3310 0x7fa8dc159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/2654960905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc06cec0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 -- 172.21.15.111:0/2654960905 >> 172.21.15.111:0/2654960905 conn(0x7fa8dc06a4c0 msgr2=0x7fa8dc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:31.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 -- 172.21.15.111:0/2654960905 shutdown_connections 2023-12-04T05:39:31.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.573+0000 7fa8e3e56640 1 -- 172.21.15.111:0/2654960905 wait complete. 2023-12-04T05:39:31.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.574+0000 7fa8e3e56640 1 Processor -- start 2023-12-04T05:39:31.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.574+0000 7fa8e3e56640 1 -- start start 2023-12-04T05:39:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e3e56640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8dc06c250 0x7fa8dc157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e3e56640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e3e56640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 0x7fa8dc1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e3e56640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc157f00 con 0x7fa8dc0b3310 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e3e56640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1581e0 con 0x7fa8dc06cae0 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e3e56640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1584c0 con 0x7fa8dc06c250 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e23f3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 0x7fa8dc1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e23f3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 0x7fa8dc1446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43478/0 (socket says 172.21.15.111:43478) 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e23f3640 1 -- 172.21.15.111:0/1511103011 learned_addr learned my addr 172.21.15.111:0/1511103011 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e1bf2640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8dc06c250 0x7fa8dc157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.575+0000 7fa8e13f1640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e23f3640 1 -- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8dc06c250 msgr2=0x7fa8dc157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e23f3640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8dc06c250 0x7fa8dc157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e23f3640 1 -- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8dc06cae0 msgr2=0x7fa8dc14df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e23f3640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc14df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e23f3640 1 -- 172.21.15.111:0/1511103011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009b30 con 0x7fa8dc0b3310 2023-12-04T05:39:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e1bf2640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8dc06c250 0x7fa8dc157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e23f3640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 0x7fa8dc1446c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4001eb0 tx=0x7fa8d400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40074e0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4007640 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc1587a0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40049e0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.576+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc14d320 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.577+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc158cc0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.577+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa8d4004b40 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.578+0000 7fa8d27fc640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa8c006b2a0 0x7fa8c006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.578+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8d4089370 con 0x7fa8dc0b3310 2023-12-04T05:39:31.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.578+0000 7fa8e1bf2640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa8c006b2a0 0x7fa8c006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:31.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.578+0000 7fa8e1bf2640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa8c006b2a0 0x7fa8c006d760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8035000 tx=0x7fa8d8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:31.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.582+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8dc158cc0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.783+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8dc158cc0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.783+0000 7fa8d27fc640 1 -- 172.21.15.111:0/1511103011 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa8dc158cc0 con 0x7fa8dc0b3310 2023-12-04T05:39:31.785 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa8c006b2a0 msgr2=0x7fa8c006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa8c006b2a0 0x7fa8c006d760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8035000 tx=0x7fa8d8030000 comp rx=0 tx=0).stop 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 msgr2=0x7fa8dc1446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 0x7fa8dc1446c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4001eb0 tx=0x7fa8d400c820 comp rx=0 tx=0).stop 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 shutdown_connections 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa8c006b2a0 0x7fa8c006d760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc0b3310 0x7fa8dc1446c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8dc06cae0 0x7fa8dc14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 --2- 172.21.15.111:0/1511103011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8dc06c250 0x7fa8dc157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:31.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 >> 172.21.15.111:0/1511103011 conn(0x7fa8dc06a4c0 msgr2=0x7fa8dc1486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:31.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 shutdown_connections 2023-12-04T05:39:31.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:31.785+0000 7fa8e3e56640 1 -- 172.21.15.111:0/1511103011 wait complete. 2023-12-04T05:39:31.797 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:39:31.797 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:39:31.798 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-04T05:39:31.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:31.798 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:31.813 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42146sec preferred_lft 42146sec 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:31.814 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8e:82:5e:fe:4f:bf brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::142c:d5ff:fef0:ffc5/64 scope link 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8e:82:5e:fe:4f:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::8c82:5eff:fefe:4fbf/64 scope link tentative 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:31.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:31.815 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:39:31.878 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-12-04T05:39:31.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:31.880 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-12-04T05:39:31.936 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-12-04T05:39:31.936 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:39:31.936 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:39:31.936 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:39:31.936 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:39:31.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:31.937 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:31.937 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:31.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:39:31.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:39:31.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:39:31.937 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:32.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.032 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:39:32.046 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-04T05:39:32.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.046 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-12-04T05:39:32.100 INFO:teuthology.orchestra.run.smithi111.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-04T05:39:32.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.101 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:32.101 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:32.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:39:32.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-04T05:39:32.101 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:32.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.249 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:39:32.250 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether ea:da:ce:ea:d2:81 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-04T05:39:32.250 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-04T05:39:32.250 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.250 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::e8da:ceff:feea:d281/64 scope link tentative 2023-12-04T05:39:32.250 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-04T05:39:32.317 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether ea:da:ce:ea:d2:81 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::e8da:ceff:feea:d281/64 scope link tentative 2023-12-04T05:39:32.318 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:32.319 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-04T05:39:32.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-04T05:39:32.320 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:32.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.443 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:39:32.443 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:39:32.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.1 up 2023-12-04T05:39:32.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.1 master ceph-brx 2023-12-04T05:39:32.443 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:39:32.485 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-12-04T05:39:32.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:32.485 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:32.542 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-04T05:39:32.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:32.543 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:32.596 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:39:32.665 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:39:32.721 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:39:32.722 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:39:32.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.723 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:39:32.776 INFO:teuthology.orchestra.run.smithi111.stdout:41 2023-12-04T05:39:32.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-12-04T05:39:32.776 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-12-04T05:39:32.819 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:39:32.845 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:39:32.901 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:32.902 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:32.903 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:39:32.904 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:39:32.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:32.904 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:39:32.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:2023-12-04T05:39:32.902+0000 7faefaa3e480 -1 init, newargv = 0x55b5882de530 newargc=15 2023-12-04T05:39:32.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:ceph-fuse[98526]: starting ceph client 2023-12-04T05:39:32.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:ceph-fuse[98526]: starting fuse 2023-12-04T05:39:32.920 INFO:teuthology.orchestra.run.smithi111.stdout:41 2023-12-04T05:39:32.920 INFO:teuthology.orchestra.run.smithi111.stdout:44 2023-12-04T05:39:32.920 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-12-04T05:39:32.921 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.2")) 2023-12-04T05:39:32.922 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-12-04T05:39:33.039 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.2.98526.asok 2023-12-04T05:39:33.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:39:33.049 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.98526.asok 2023-12-04T05:39:33.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:33.050 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.98526.asok status 2023-12-04T05:39:33.185 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:39:33.185 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-12-04T05:39:33.185 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:39:33.185 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:39:33.185 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "2", 2023-12-04T05:39:33.185 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "98526", 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4734, 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4734 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:39:33.186 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 582052814 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 582052814 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4734 192.168.0.2:0/582052814", 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.2:0/582052814", 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 6, 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 20, 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:39:33.187 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-12-04T05:39:33.188 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-12-04T05:39:33.188 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:39:33.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:39:33.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.293+0000 7f23ee8da640 1 Processor -- start 2023-12-04T05:39:33.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.293+0000 7f23ee8da640 1 -- start start 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ee8da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8146a20 0x7f23e80b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ee8da640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23e806a4c0 0x7f23e806a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ee8da640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e815db90 con 0x7f23e8071500 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ee8da640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e806ae90 con 0x7f23e806a4c0 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ee8da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80b3650 con 0x7f23e8146a20 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ece77640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23e806a4c0 0x7f23e806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:33.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ece77640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f23e806a4c0 0x7f23e806a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43484/0 (socket says 172.21.15.111:43484) 2023-12-04T05:39:33.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.294+0000 7f23ece77640 1 -- 172.21.15.111:0/4131823540 learned_addr learned my addr 172.21.15.111:0/4131823540 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:33.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23e7fff640 1 --2- 172.21.15.111:0/4131823540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8146a20 0x7f23e80b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:33.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23ece77640 1 -- 172.21.15.111:0/4131823540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8146a20 msgr2=0x7f23e80b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:33.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23ece77640 1 --2- 172.21.15.111:0/4131823540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8146a20 0x7f23e80b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23ece77640 1 -- 172.21.15.111:0/4131823540 >> v1:172.21.15.192:6789/0 conn(0x7f23e8071500 legacy=0x7f23e815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:33.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23ece77640 1 -- 172.21.15.111:0/4131823540 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e815d840 con 0x7f23e806a4c0 2023-12-04T05:39:33.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23ece77640 1 --2- 172.21.15.111:0/4131823540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e806a4c0 0x7f23e806a950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009e50 tx=0x7f23e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4103e463c938e4e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:33.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23e6ffd640 1 -- 172.21.15.111:0/4131823540 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e002c9e0 con 0x7f23e806a4c0 2023-12-04T05:39:33.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23e6ffd640 1 -- 172.21.15.111:0/4131823540 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e002cb40 con 0x7f23e806a4c0 2023-12-04T05:39:33.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.295+0000 7f23e6ffd640 1 -- 172.21.15.111:0/4131823540 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00343f0 con 0x7f23e806a4c0 2023-12-04T05:39:33.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 -- 172.21.15.111:0/4131823540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e806a4c0 msgr2=0x7f23e806a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 --2- 172.21.15.111:0/4131823540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e806a4c0 0x7f23e806a950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009e50 tx=0x7f23e002be40 comp rx=0 tx=0).stop 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 -- 172.21.15.111:0/4131823540 shutdown_connections 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 --2- 172.21.15.111:0/4131823540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e806a4c0 0x7f23e806a950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 --2- 172.21.15.111:0/4131823540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8146a20 0x7f23e80b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 -- 172.21.15.111:0/4131823540 >> 172.21.15.111:0/4131823540 conn(0x7f23e806b5d0 msgr2=0x7f23e806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 -- 172.21.15.111:0/4131823540 shutdown_connections 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.296+0000 7f23ee8da640 1 -- 172.21.15.111:0/4131823540 wait complete. 2023-12-04T05:39:33.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.297+0000 7f23ee8da640 1 Processor -- start 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.297+0000 7f23ee8da640 1 -- start start 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ee8da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23e806a4c0 0x7f23e8143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ee8da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8071500 0x7f23e81f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ee8da640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 0x7f23e81f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ee8da640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8140f00 con 0x7f23e8146a20 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ee8da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81e62d0 con 0x7f23e806a4c0 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ee8da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81e65b0 con 0x7f23e8071500 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ece77640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 0x7f23e81f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ece77640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 0x7f23e81f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43494/0 (socket says 172.21.15.111:43494) 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23e7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23e806a4c0 0x7f23e8143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23ece77640 1 -- 172.21.15.111:0/2471751122 learned_addr learned my addr 172.21.15.111:0/2471751122 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:33.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.298+0000 7f23e77fe640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8071500 0x7f23e81f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ece77640 1 -- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8071500 msgr2=0x7f23e81f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ece77640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8071500 0x7f23e81f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ece77640 1 -- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23e806a4c0 msgr2=0x7f23e8143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ece77640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23e806a4c0 0x7f23e8143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ece77640 1 -- 172.21.15.111:0/2471751122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0009b30 con 0x7f23e8146a20 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ece77640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 0x7f23e81f69e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23e0036040 tx=0x7f23e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e003e450 con 0x7f23e8146a20 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e003e5d0 con 0x7f23e8146a20 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e81e6890 con 0x7f23e8146a20 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e003e980 con 0x7f23e8146a20 2023-12-04T05:39:33.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.299+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e81e6e50 con 0x7f23e8146a20 2023-12-04T05:39:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.300+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e81e6fe0 con 0x7f23e8146a20 2023-12-04T05:39:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.300+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f23e003eae0 con 0x7f23e8146a20 2023-12-04T05:39:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.301+0000 7f23e4ff9640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23d406b2a0 0x7f23d406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.301+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23e00ac940 con 0x7f23e8146a20 2023-12-04T05:39:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.301+0000 7f23e7fff640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23d406b2a0 0x7f23d406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.301+0000 7f23e7fff640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23d406b2a0 0x7f23d406d760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23d800aac0 tx=0x7f23d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:33.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.305+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23e0077860 con 0x7f23e8146a20 2023-12-04T05:39:33.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.506+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23e80693d0 con 0x7f23e8146a20 2023-12-04T05:39:33.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.506+0000 7f23e4ff9640 1 -- 172.21.15.111:0/2471751122 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f23e0043060 con 0x7f23e8146a20 2023-12-04T05:39:33.508 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23d406b2a0 msgr2=0x7f23d406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23d406b2a0 0x7f23d406d760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f23d800aac0 tx=0x7f23d8008040 comp rx=0 tx=0).stop 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 msgr2=0x7f23e81f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 0x7f23e81f69e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23e0036040 tx=0x7f23e0004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 shutdown_connections 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f23d406b2a0 0x7f23d406d760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23e8146a20 0x7f23e81f69e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23e8071500 0x7f23e81f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 --2- 172.21.15.111:0/2471751122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23e806a4c0 0x7f23e8143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 >> 172.21.15.111:0/2471751122 conn(0x7f23e806b5d0 msgr2=0x7f23e8071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 shutdown_connections 2023-12-04T05:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:33.508+0000 7f23ee8da640 1 -- 172.21.15.111:0/2471751122 wait complete. 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:39:33.520 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:39:33.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:33.521 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:33.539 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42133sec preferred_lft 42133sec 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-04T05:39:33.540 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:33.541 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-04T05:39:33.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-04T05:39:33.541 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:33.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:33.708 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:39:33.735 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-04T05:39:33.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:33.737 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-04T05:39:33.811 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-04T05:39:33.811 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:39:33.811 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:39:33.811 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:39:33.811 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:39:33.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:33.812 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:33.812 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:33.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:39:33.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:39:33.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:39:33.812 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:34.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.100 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-04T05:39:34.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.115 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2023-12-04T05:39:34.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.170 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:34.170 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:34.170 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:34.170 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-04T05:39:34.170 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:34.254 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-04T05:39:34.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-04T05:39:34.255 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:34.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.351 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:34.351 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:34.351 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-04T05:39:34.351 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2023-12-04T05:39:34.351 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:34.393 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-04T05:39:34.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:34.393 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:34.452 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:39:34.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:34.452 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:34.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:39:34.575 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:34.630 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:39:34.630 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:39:34.630 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:39:34.630 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:39:34.631 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:39:34.632 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:39:34.633 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:34.633 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:34.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.633 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:39:34.685 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:39:34.686 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-04T05:39:34.729 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:39:34.755 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:39:34.811 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:39:34.812 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:34.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:34.813 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:39:34.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-04T05:39:34.850+0000 7f6810842480 -1 init, newargv = 0x56131a0595a0 newargc=15 2023-12-04T05:39:34.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[95322]: starting ceph client 2023-12-04T05:39:34.856 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[95322]: starting fuse 2023-12-04T05:39:34.867 INFO:teuthology.orchestra.run.smithi192.stdout:41 2023-12-04T05:39:34.867 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:39:34.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:39:34.867 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-04T05:39:34.867 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-04T05:39:34.867 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-04T05:39:34.867 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-04T05:39:34.867 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-04T05:39:34.868 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-04T05:39:34.869 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-04T05:39:34.869 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:39:34.869 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:34.869 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-04T05:39:34.869 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-04T05:39:34.988 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.95322.asok 2023-12-04T05:39:34.991 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:39:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:39:34.999 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95322.asok 2023-12-04T05:39:34.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.000 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95322.asok status 2023-12-04T05:39:35.147 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "95322", 2023-12-04T05:39:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4600, 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4600 2023-12-04T05:39:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 4045952157 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:39:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 4045952157 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4600 192.168.0.1:0/4045952157", 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/4045952157", 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 6, 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-04T05:39:35.151 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-04T05:39:35.156 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:39:35.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.266+0000 7fc84ec3d640 1 Processor -- start 2023-12-04T05:39:35.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.266+0000 7fc84ec3d640 1 -- start start 2023-12-04T05:39:35.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.266+0000 7fc84ec3d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc848143910 0x7fc84806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:35.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84ec3d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8480b3ea0 con 0x7fc8480b3610 2023-12-04T05:39:35.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84ec3d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8480b3bc0 con 0x7fc8480713c0 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84ec3d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84806d700 con 0x7fc848143910 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84d1da640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc8480b3610 0x7fc8480cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51066/0 (socket says 172.21.15.111:51066) 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84d1da640 1 -- 172.21.15.111:0/2534142541 learned_addr learned my addr 172.21.15.111:0/2534142541 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84c9d9640 1 --2- 172.21.15.111:0/2534142541 >> v2:172.21.15.192:3300/0 conn(0x7fc848143910 0x7fc84806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc8477fe640 1 -- 172.21.15.111:0/2534142541 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296175434 0 0) 0x7fc8480b3ea0 con 0x7fc8480b3610 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc8477fe640 1 -- 172.21.15.111:0/2534142541 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc838003520 con 0x7fc8480b3610 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84c9d9640 1 -- 172.21.15.111:0/2534142541 >> v1:172.21.15.192:6789/0 conn(0x7fc8480713c0 legacy=0x7fc8480b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84c9d9640 1 -- 172.21.15.111:0/2534142541 >> v1:172.21.15.111:6789/0 conn(0x7fc8480b3610 legacy=0x7fc8480cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.267+0000 7fc84c9d9640 1 -- 172.21.15.111:0/2534142541 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84806d9e0 con 0x7fc848143910 2023-12-04T05:39:35.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.268+0000 7fc84c9d9640 1 --2- 172.21.15.111:0/2534142541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc84806e1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097a0 tx=0x7fc83c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab07667439d03b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.268+0000 7fc8477fe640 1 -- 172.21.15.111:0/2534142541 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c030610 con 0x7fc848143910 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.268+0000 7fc8477fe640 1 -- 172.21.15.111:0/2534142541 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c030770 con 0x7fc848143910 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.268+0000 7fc8477fe640 1 -- 172.21.15.111:0/2534142541 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c030b00 con 0x7fc848143910 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 -- 172.21.15.111:0/2534142541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 msgr2=0x7fc84806e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/2534142541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc84806e1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097a0 tx=0x7fc83c02b960 comp rx=0 tx=0).stop 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 -- 172.21.15.111:0/2534142541 shutdown_connections 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/2534142541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc84806e1e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 -- 172.21.15.111:0/2534142541 >> 172.21.15.111:0/2534142541 conn(0x7fc8480b1c00 msgr2=0x7fc8480b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 -- 172.21.15.111:0/2534142541 shutdown_connections 2023-12-04T05:39:35.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.269+0000 7fc84ec3d640 1 -- 172.21.15.111:0/2534142541 wait complete. 2023-12-04T05:39:35.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.270+0000 7fc84ec3d640 1 Processor -- start 2023-12-04T05:39:35.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.270+0000 7fc84ec3d640 1 -- start start 2023-12-04T05:39:35.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84ec3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8480713c0 0x7fc8480a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:35.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84ec3d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 0x7fc848159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84ec3d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc848159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84ec3d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480ca390 con 0x7fc8480b3610 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84ec3d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480c9710 con 0x7fc848143910 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84ec3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8480c99f0 con 0x7fc8480713c0 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 0x7fc848159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 0x7fc848159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43504/0 (socket says 172.21.15.111:43504) 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 -- 172.21.15.111:0/4109487489 learned_addr learned my addr 172.21.15.111:0/4109487489 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84c9d9640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8480713c0 0x7fc8480a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84d1da640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc848159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 -- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8480713c0 msgr2=0x7fc8480a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8480713c0 0x7fc8480a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 -- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 msgr2=0x7fc848159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc848159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc847fff640 1 -- 172.21.15.111:0/4109487489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c009400 con 0x7fc8480b3610 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84c9d9640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8480713c0 0x7fc8480a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.271+0000 7fc84d1da640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc848159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.272+0000 7fc847fff640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 0x7fc848159950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc83400c450 tx=0x7fc83400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.272+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8340052d0 con 0x7fc8480b3610 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.272+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc834005430 con 0x7fc8480b3610 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.272+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8480c9cd0 con 0x7fc8480b3610 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.272+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834012870 con 0x7fc8480b3610 2023-12-04T05:39:35.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.272+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8480a8c10 con 0x7fc8480b3610 2023-12-04T05:39:35.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.273+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8480ca4d0 con 0x7fc8480b3610 2023-12-04T05:39:35.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.273+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc8340129d0 con 0x7fc8480b3610 2023-12-04T05:39:35.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.274+0000 7fc8457fa640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc82c06b2a0 0x7fc82c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:35.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.274+0000 7fc84c9d9640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc82c06b2a0 0x7fc82c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:35.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.274+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc834089670 con 0x7fc8480b3610 2023-12-04T05:39:35.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.274+0000 7fc84c9d9640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc82c06b2a0 0x7fc82c06d760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc83c036000 tx=0x7fc83c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:35.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.278+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8480ca4d0 con 0x7fc8480b3610 2023-12-04T05:39:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.480+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8480ca4d0 con 0x7fc8480b3610 2023-12-04T05:39:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.480+0000 7fc8457fa640 1 -- 172.21.15.111:0/4109487489 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc8480ca4d0 con 0x7fc8480b3610 2023-12-04T05:39:35.482 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc82c06b2a0 msgr2=0x7fc82c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc82c06b2a0 0x7fc82c06d760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc83c036000 tx=0x7fc83c03b040 comp rx=0 tx=0).stop 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 msgr2=0x7fc848159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 0x7fc848159950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc83400c450 tx=0x7fc83400c910 comp rx=0 tx=0).stop 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 shutdown_connections 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc82c06b2a0 0x7fc82c06d760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc848143910 0x7fc848159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8480b3610 0x7fc848159950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 --2- 172.21.15.111:0/4109487489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8480713c0 0x7fc8480a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:35.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 >> 172.21.15.111:0/4109487489 conn(0x7fc8480b1c00 msgr2=0x7fc8480cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:35.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 shutdown_connections 2023-12-04T05:39:35.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:35.482+0000 7fc84ec3d640 1 -- 172.21.15.111:0/4109487489 wait complete. 2023-12-04T05:39:35.494 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:39:35.494 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:39:35.494 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-04T05:39:35.494 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-04T05:39:35.494 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:35.495 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:39:35.495 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:39:35.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:39:35.495 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-04T05:39:35.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.495 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:35.511 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42131sec preferred_lft 42131sec 2023-12-04T05:39:35.512 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 1a:e1:a2:48:a2:71 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::495:82ff:fe57:3769/64 scope link 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 1a:e1:a2:48:a2:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::18e1:a2ff:fe48:a271/64 scope link tentative 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.513 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:39:35.577 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-04T05:39:35.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.578 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-04T05:39:35.635 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-04T05:39:35.635 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:39:35.635 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:39:35.635 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:39:35.635 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:39:35.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.636 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:35.636 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:35.636 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:39:35.636 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:39:35.636 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:39:35.637 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:35.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.731 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-04T05:39:35.745 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-04T05:39:35.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.746 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2023-12-04T05:39:35.800 INFO:teuthology.orchestra.run.smithi192.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-04T05:39:35.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.800 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:35.800 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:35.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:39:35.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-04T05:39:35.800 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:35.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.882 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-04T05:39:35.948 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:35.948 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether fa:3e:35:41:0b:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::f83e:35ff:fe41:b40/64 scope link 2023-12-04T05:39:35.949 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:35.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:35.950 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether fa:3e:35:41:0b:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-04T05:39:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::f83e:35ff:fe41:b40/64 scope link 2023-12-04T05:39:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:39:36.020 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-04T05:39:36.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-04T05:39:36.020 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:36.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.145 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:39:36.145 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:39:36.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.1 up 2023-12-04T05:39:36.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.1 master ceph-brx 2023-12-04T05:39:36.145 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:39:36.186 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-12-04T05:39:36.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:36.186 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:36.241 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-04T05:39:36.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:39:36.241 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:36.296 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:39:36.363 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:39:36.419 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:39:36.420 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:39:36.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.422 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:39:36.473 INFO:teuthology.orchestra.run.smithi192.stdout:41 2023-12-04T05:39:36.474 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-12-04T05:39:36.474 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-12-04T05:39:36.516 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:39:36.543 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:39:36.599 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:39:36.600 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:39:36.601 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:39:36.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.602 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:39:36.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:2023-12-04T05:39:36.601+0000 7ff787a1f480 -1 init, newargv = 0x555a0a2da650 newargc=15 2023-12-04T05:39:36.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[95757]: starting ceph client 2023-12-04T05:39:36.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[95757]: starting fuse 2023-12-04T05:39:36.618 INFO:teuthology.orchestra.run.smithi192.stdout:41 2023-12-04T05:39:36.618 INFO:teuthology.orchestra.run.smithi192.stdout:44 2023-12-04T05:39:36.618 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-12-04T05:39:36.618 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:39:36.618 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-04T05:39:36.618 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-04T05:39:36.618 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:39:36.619 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.3")) 2023-12-04T05:39:36.620 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-04T05:39:36.736 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.3.95757.asok 2023-12-04T05:39:36.739 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:39:36.746 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.95757.asok 2023-12-04T05:39:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.95757.asok status 2023-12-04T05:39:36.881 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-04T05:39:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-04T05:39:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "3", 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "95757", 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-04T05:39:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-04T05:39:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-04T05:39:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4612, 2023-12-04T05:39:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-04T05:39:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-04T05:39:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4612 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 71367090 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 71367090 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4612 192.168.0.2:0/71367090", 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.2:0/71367090", 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 6, 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-04T05:39:36.885 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-04T05:39:36.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.890 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-12-04T05:39:36.891 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:36.909 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-12-04T05:39:36.909 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:36.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:36.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:36.981 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-12-04T05:39:36.982 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:37.038 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-12-04T05:39:37.039 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:37.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:37.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:37.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:37.111 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-04T05:39:37.111 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:37.127 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-04T05:39:37.127 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:37.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:37.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:37.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:37.195 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-04T05:39:37.196 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:37.252 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-04T05:39:37.252 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:37.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:37.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:37.319 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-04T05:39:37.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:37.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 Processor -- start 2023-12-04T05:39:37.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 -- start start 2023-12-04T05:39:37.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b980b20a0 0x7f1b980d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1b980b2ad0 0x7f1b980b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b980cdcb0 con 0x7f1b980d1780 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980d39e0 con 0x7f1b980b20a0 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.442+0000 7f1b9e81e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980d3cc0 con 0x7f1b980b2ad0 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b980b20a0 0x7f1b980d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b980b20a0 0x7f1b980d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43518/0 (socket says 172.21.15.111:43518) 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 -- 172.21.15.111:0/1403001293 learned_addr learned my addr 172.21.15.111:0/1403001293 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b9cdbb640 1 --2- 172.21.15.111:0/1403001293 >> v2:172.21.15.192:3300/0 conn(0x7f1b980b2ad0 0x7f1b980b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 -- 172.21.15.111:0/1403001293 >> v2:172.21.15.192:3300/0 conn(0x7f1b980b2ad0 msgr2=0x7f1b980b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 --2- 172.21.15.111:0/1403001293 >> v2:172.21.15.192:3300/0 conn(0x7f1b980b2ad0 0x7f1b980b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 -- 172.21.15.111:0/1403001293 >> v1:172.21.15.192:6789/0 conn(0x7f1b980d1780 legacy=0x7f1b980d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:37.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.443+0000 7f1b977fe640 1 -- 172.21.15.111:0/1403001293 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980cdf50 con 0x7f1b980b20a0 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b977fe640 1 --2- 172.21.15.111:0/1403001293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980d01e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b980d1680 tx=0x7f1b8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9788da20d5305785 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b96ffd640 1 -- 172.21.15.111:0/1403001293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0317e0 con 0x7f1b980b20a0 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b96ffd640 1 -- 172.21.15.111:0/1403001293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c031940 con 0x7f1b980b20a0 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b96ffd640 1 -- 172.21.15.111:0/1403001293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c031cd0 con 0x7f1b980b20a0 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b9e81e640 1 -- 172.21.15.111:0/1403001293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 msgr2=0x7f1b980d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/1403001293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980d01e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b980d1680 tx=0x7f1b8c02b880 comp rx=0 tx=0).stop 2023-12-04T05:39:37.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b9e81e640 1 -- 172.21.15.111:0/1403001293 shutdown_connections 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/1403001293 >> v2:172.21.15.192:3300/0 conn(0x7f1b980b2ad0 0x7f1b980b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/1403001293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980d01e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.444+0000 7f1b9e81e640 1 -- 172.21.15.111:0/1403001293 >> 172.21.15.111:0/1403001293 conn(0x7f1b9809c4e0 msgr2=0x7f1b980d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.445+0000 7f1b9e81e640 1 -- 172.21.15.111:0/1403001293 shutdown_connections 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.445+0000 7f1b9e81e640 1 -- 172.21.15.111:0/1403001293 wait complete. 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.446+0000 7f1b9e81e640 1 Processor -- start 2023-12-04T05:39:37.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.446+0000 7f1b9e81e640 1 -- start start 2023-12-04T05:39:37.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b980b2ad0 0x7f1b980b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b980d1780 0x7f1b980b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98157940 con 0x7f1b980b20a0 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98157c20 con 0x7f1b980b2ad0 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98157f00 con 0x7f1b980d1780 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43522/0 (socket says 172.21.15.111:43522) 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 -- 172.21.15.111:0/3753964554 learned_addr learned my addr 172.21.15.111:0/3753964554 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9cdbb640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b980d1780 0x7f1b980b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b977fe640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b980b2ad0 0x7f1b980b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 -- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b980d1780 msgr2=0x7f1b980b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b980d1780 0x7f1b980b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 -- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b980b2ad0 msgr2=0x7f1b980b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b980b2ad0 0x7f1b980b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 -- 172.21.15.111:0/3753964554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0094d0 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b977fe640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b980b2ad0 0x7f1b980b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b97fff640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980b5dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b3f0 tx=0x7f1b8800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88011830 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88011990 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b981581e0 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88011d40 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.447+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b981575f0 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.448+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98158700 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.449+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1b8801a400 con 0x7f1b980b20a0 2023-12-04T05:39:37.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.449+0000 7f1b94ff9640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1b8406b2a0 0x7f1b8406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.449+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b88089ca0 con 0x7f1b980b20a0 2023-12-04T05:39:37.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.449+0000 7f1b977fe640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1b8406b2a0 0x7f1b8406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.450+0000 7f1b977fe640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1b8406b2a0 0x7f1b8406d760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c036000 tx=0x7f1b8c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:37.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.454+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b98158700 con 0x7f1b980b20a0 2023-12-04T05:39:37.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.621+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b98158700 con 0x7f1b980b20a0 2023-12-04T05:39:37.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.622+0000 7f1b94ff9640 1 -- 172.21.15.111:0/3753964554 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f1b98158700 con 0x7f1b980b20a0 2023-12-04T05:39:37.624 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:37.624 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:37.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1b8406b2a0 msgr2=0x7f1b8406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1b8406b2a0 0x7f1b8406d760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c036000 tx=0x7f1b8c03d040 comp rx=0 tx=0).stop 2023-12-04T05:39:37.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 msgr2=0x7f1b980b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980b5dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b3f0 tx=0x7f1b8800b8b0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 shutdown_connections 2023-12-04T05:39:37.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1b8406b2a0 0x7f1b8406d760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1b980d1780 0x7f1b980b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b980b2ad0 0x7f1b980b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 --2- 172.21.15.111:0/3753964554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b980b20a0 0x7f1b980b5dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.624+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 >> 172.21.15.111:0/3753964554 conn(0x7f1b9809c4e0 msgr2=0x7f1b9815a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:37.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.625+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 shutdown_connections 2023-12-04T05:39:37.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.625+0000 7f1b9e81e640 1 -- 172.21.15.111:0/3753964554 wait complete. 2023-12-04T05:39:37.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:37.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:37.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 Processor -- start 2023-12-04T05:39:37.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 -- start start 2023-12-04T05:39:37.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f379413f640 0x7f379413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3794071d20 0x7f3794072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379413dd10 con 0x7f379413cee0 2023-12-04T05:39:37.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3794072670 con 0x7f379413f640 2023-12-04T05:39:37.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.738+0000 7f379ab7c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379413d2c0 con 0x7f3794071d20 2023-12-04T05:39:37.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f379413f640 0x7f379413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3799119640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3794071d20 0x7f3794072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f379413f640 0x7f379413fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43530/0 (socket says 172.21.15.111:43530) 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 -- 172.21.15.111:0/3095951471 learned_addr learned my addr 172.21.15.111:0/3095951471 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3798918640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f379413cee0 0x7f379413d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43538/0 (socket says 172.21.15.111:43538) 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 -- 172.21.15.111:0/3095951471 >> v2:172.21.15.192:3300/0 conn(0x7f3794071d20 msgr2=0x7f3794072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 --2- 172.21.15.111:0/3095951471 >> v2:172.21.15.192:3300/0 conn(0x7f3794071d20 0x7f3794072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 -- 172.21.15.111:0/3095951471 >> v1:172.21.15.192:6789/0 conn(0x7f379413cee0 legacy=0x7f379413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.739+0000 7f3793fff640 1 -- 172.21.15.111:0/3095951471 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37941400e0 con 0x7f379413f640 2023-12-04T05:39:37.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f3793fff640 1 --2- 172.21.15.111:0/3095951471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f379413fab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f379406bdd0 tx=0x7f378002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec1e61e7cd3922 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f37937fe640 1 -- 172.21.15.111:0/3095951471 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37800317e0 con 0x7f379413f640 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f37937fe640 1 -- 172.21.15.111:0/3095951471 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780031940 con 0x7f379413f640 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f37937fe640 1 -- 172.21.15.111:0/3095951471 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780031cd0 con 0x7f379413f640 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 -- 172.21.15.111:0/3095951471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 msgr2=0x7f379413fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3095951471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f379413fab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f379406bdd0 tx=0x7f378002b880 comp rx=0 tx=0).stop 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 -- 172.21.15.111:0/3095951471 shutdown_connections 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3095951471 >> v2:172.21.15.192:3300/0 conn(0x7f3794071d20 0x7f3794072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3095951471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f379413fab0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 -- 172.21.15.111:0/3095951471 >> 172.21.15.111:0/3095951471 conn(0x7f37940b0cf0 msgr2=0x7f37940b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 -- 172.21.15.111:0/3095951471 shutdown_connections 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.740+0000 7f379ab7c640 1 -- 172.21.15.111:0/3095951471 wait complete. 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.741+0000 7f379ab7c640 1 Processor -- start 2023-12-04T05:39:37.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.741+0000 7f379ab7c640 1 -- start start 2023-12-04T05:39:37.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f379ab7c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3794071d20 0x7f3794136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f379ab7c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379413cee0 0x7f37940b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f379ab7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f37940b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f379ab7c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379414f780 con 0x7f379413f640 2023-12-04T05:39:37.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f379ab7c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940b4330 con 0x7f3794071d20 2023-12-04T05:39:37.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f379ab7c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940b4600 con 0x7f379413cee0 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f3799119640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f37940b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f3799119640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f37940b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43542/0 (socket says 172.21.15.111:43542) 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.742+0000 7f3799119640 1 -- 172.21.15.111:0/3651365356 learned_addr learned my addr 172.21.15.111:0/3651365356 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3798918640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3794071d20 0x7f3794136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3793fff640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379413cee0 0x7f37940b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3799119640 1 -- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379413cee0 msgr2=0x7f37940b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3799119640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379413cee0 0x7f37940b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3799119640 1 -- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3794071d20 msgr2=0x7f3794136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3799119640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3794071d20 0x7f3794136930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3799119640 1 -- 172.21.15.111:0/3651365356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800094d0 con 0x7f379413f640 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f3799119640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f37940b3df0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f378c00efc0 tx=0x7f378c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c014480 con 0x7f379413f640 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c00f040 con 0x7f379413f640 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940b48e0 con 0x7f379413f640 2023-12-04T05:39:37.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c014820 con 0x7f379413f640 2023-12-04T05:39:37.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.743+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379414f410 con 0x7f379413f640 2023-12-04T05:39:37.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.744+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379414f8f0 con 0x7f379413f640 2023-12-04T05:39:37.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.744+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f378c014980 con 0x7f379413f640 2023-12-04T05:39:37.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.745+0000 7f37917fa640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f377806b2d0 0x7f377806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:37.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.745+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f378c08a670 con 0x7f379413f640 2023-12-04T05:39:37.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.745+0000 7f3798918640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f377806b2d0 0x7f377806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:37.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.745+0000 7f3798918640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f377806b2d0 0x7f377806d790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3788007fd0 tx=0x7f3788007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:37.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.749+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f378c055500 con 0x7f379413f640 2023-12-04T05:39:37.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.915+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3794069360 con 0x7f379413f640 2023-12-04T05:39:37.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.915+0000 7f37917fa640 1 -- 172.21.15.111:0/3651365356 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f378c018020 con 0x7f379413f640 2023-12-04T05:39:37.917 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:37.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f377806b2d0 msgr2=0x7f377806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f377806b2d0 0x7f377806d790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3788007fd0 tx=0x7f3788007e00 comp rx=0 tx=0).stop 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 msgr2=0x7f37940b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f37940b3df0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f378c00efc0 tx=0x7f378c00c490 comp rx=0 tx=0).stop 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 shutdown_connections 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f377806b2d0 0x7f377806d790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379413f640 0x7f37940b3df0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379413cee0 0x7f37940b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 --2- 172.21.15.111:0/3651365356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3794071d20 0x7f3794136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:37.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 >> 172.21.15.111:0/3651365356 conn(0x7f37940b0cf0 msgr2=0x7f37940aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:37.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 shutdown_connections 2023-12-04T05:39:37.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:37.917+0000 7f379ab7c640 1 -- 172.21.15.111:0/3651365356 wait complete. 2023-12-04T05:39:37.929 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:38.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.030+0000 7fa64b733640 1 Processor -- start 2023-12-04T05:39:38.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.030+0000 7fa64b733640 1 -- start start 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa64b733640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64413cee0 0x7fa64413d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa64b733640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa64413f640 0x7fa64413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa64b733640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa644071d20 0x7fa644072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa64b733640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa644072690 con 0x7fa644071d20 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa64b733640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64413d2c0 con 0x7fa64413f640 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa64b733640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441400e0 con 0x7fa64413cee0 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa644071d20 0x7fa644072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa644071d20 0x7fa644072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43544/0 (socket says 172.21.15.111:43544) 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 -- 172.21.15.111:0/4139449866 learned_addr learned my addr 172.21.15.111:0/4139449866 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa6494cf640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64413cee0 0x7fa64413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa648cce640 1 --2- 172.21.15.111:0/4139449866 >> v2:172.21.15.192:3300/0 conn(0x7fa64413f640 0x7fa64413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 -- 172.21.15.111:0/4139449866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64413cee0 msgr2=0x7fa64413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64413cee0 0x7fa64413d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 -- 172.21.15.111:0/4139449866 >> v2:172.21.15.192:3300/0 conn(0x7fa64413f640 msgr2=0x7fa64413fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 --2- 172.21.15.111:0/4139449866 >> v2:172.21.15.192:3300/0 conn(0x7fa64413f640 0x7fa64413fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.031+0000 7fa649cd0640 1 -- 172.21.15.111:0/4139449866 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6441403c0 con 0x7fa644071d20 2023-12-04T05:39:38.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.032+0000 7fa6494cf640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64413cee0 0x7fa64413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.032+0000 7fa649cd0640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa644071d20 0x7fa644072150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa64406c010 tx=0x7fa63c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269800e07063cd48 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.032+0000 7fa63bfff640 1 -- 172.21.15.111:0/4139449866 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c031750 con 0x7fa644071d20 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.032+0000 7fa63bfff640 1 -- 172.21.15.111:0/4139449866 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c0318b0 con 0x7fa644071d20 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.032+0000 7fa63bfff640 1 -- 172.21.15.111:0/4139449866 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c031c40 con 0x7fa644071d20 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 -- 172.21.15.111:0/4139449866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa644071d20 msgr2=0x7fa644072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa644071d20 0x7fa644072150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa64406c010 tx=0x7fa63c02b7f0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 -- 172.21.15.111:0/4139449866 shutdown_connections 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa644071d20 0x7fa644072150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 --2- 172.21.15.111:0/4139449866 >> v2:172.21.15.192:3300/0 conn(0x7fa64413f640 0x7fa64413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 --2- 172.21.15.111:0/4139449866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64413cee0 0x7fa64413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 -- 172.21.15.111:0/4139449866 >> 172.21.15.111:0/4139449866 conn(0x7fa6440b0cf0 msgr2=0x7fa6440b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 -- 172.21.15.111:0/4139449866 shutdown_connections 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.033+0000 7fa64b733640 1 -- 172.21.15.111:0/4139449866 wait complete. 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.034+0000 7fa64b733640 1 Processor -- start 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.034+0000 7fa64b733640 1 -- start start 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa64b733640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa644071d20 0x7fa6440aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa64b733640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 0x7fa6440af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa64b733640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64413f640 0x7fa644164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa64b733640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644165b70 con 0x7fa64413cee0 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa64b733640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644165e50 con 0x7fa64413f640 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa64b733640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa644166130 con 0x7fa644071d20 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 0x7fa6440af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 0x7fa6440af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43560/0 (socket says 172.21.15.111:43560) 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 -- 172.21.15.111:0/1986851445 learned_addr learned my addr 172.21.15.111:0/1986851445 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa649cd0640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64413f640 0x7fa644164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa6494cf640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa644071d20 0x7fa6440aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 -- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa644071d20 msgr2=0x7fa6440aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa644071d20 0x7fa6440aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 -- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64413f640 msgr2=0x7fa644164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64413f640 0x7fa644164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 -- 172.21.15.111:0/1986851445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0094d0 con 0x7fa64413cee0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa6494cf640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa644071d20 0x7fa6440aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa649cd0640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64413f640 0x7fa644164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.035+0000 7fa648cce640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 0x7fa6440af3b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e9b0 tx=0x7fa63400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.036+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634009670 con 0x7fa64413cee0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.036+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63400c9a0 con 0x7fa64413cee0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.036+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400cd50 con 0x7fa64413cee0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.036+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6441411e0 con 0x7fa64413cee0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.036+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa644136a50 con 0x7fa64413cee0 2023-12-04T05:39:38.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.037+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa644141040 con 0x7fa64413cee0 2023-12-04T05:39:38.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.037+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa634013470 con 0x7fa64413cee0 2023-12-04T05:39:38.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.037+0000 7fa639ffb640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa62806b2a0 0x7fa62806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.038+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa634089d50 con 0x7fa64413cee0 2023-12-04T05:39:38.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.038+0000 7fa6494cf640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa62806b2a0 0x7fa62806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.038+0000 7fa6494cf640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa62806b2a0 0x7fa62806d760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa640003fd0 tx=0x7fa640003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.042+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa644141040 con 0x7fa64413cee0 2023-12-04T05:39:38.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.208+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa644141040 con 0x7fa64413cee0 2023-12-04T05:39:38.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.208+0000 7fa639ffb640 1 -- 172.21.15.111:0/1986851445 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa634017020 con 0x7fa64413cee0 2023-12-04T05:39:38.210 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa62806b2a0 msgr2=0x7fa62806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa62806b2a0 0x7fa62806d760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa640003fd0 tx=0x7fa640003990 comp rx=0 tx=0).stop 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 msgr2=0x7fa6440af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 0x7fa6440af3b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e9b0 tx=0x7fa63400ee70 comp rx=0 tx=0).stop 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 shutdown_connections 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa62806b2a0 0x7fa62806d760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64413f640 0x7fa644164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa64413cee0 0x7fa6440af3b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 --2- 172.21.15.111:0/1986851445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa644071d20 0x7fa6440aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 >> 172.21.15.111:0/1986851445 conn(0x7fa6440b0cf0 msgr2=0x7fa64413e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 shutdown_connections 2023-12-04T05:39:38.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.210+0000 7fa64b733640 1 -- 172.21.15.111:0/1986851445 wait complete. 2023-12-04T05:39:38.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:38.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 Processor -- start 2023-12-04T05:39:38.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 -- start start 2023-12-04T05:39:38.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faf8813cee0 0x7faf8813d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf8813f640 0x7faf8813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8813d2c0 con 0x7faf88071d20 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf881400e0 con 0x7faf8813f640 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.325+0000 7faf8f6e7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf881403c0 con 0x7faf8813cee0 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf8813f640 0x7faf8813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf8813f640 0x7faf8813fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43570/0 (socket says 172.21.15.111:43570) 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 -- 172.21.15.111:0/3809820880 learned_addr learned my addr 172.21.15.111:0/3809820880 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8d483640 1 --2- 172.21.15.111:0/3809820880 >> v2:172.21.15.192:3300/0 conn(0x7faf8813cee0 0x7faf8813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 -- 172.21.15.111:0/3809820880 >> v2:172.21.15.192:3300/0 conn(0x7faf8813cee0 msgr2=0x7faf8813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 --2- 172.21.15.111:0/3809820880 >> v2:172.21.15.192:3300/0 conn(0x7faf8813cee0 0x7faf8813d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 -- 172.21.15.111:0/3809820880 >> v1:172.21.15.192:6789/0 conn(0x7faf88071d20 legacy=0x7faf88072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.326+0000 7faf8cc82640 1 -- 172.21.15.111:0/3809820880 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880cdb50 con 0x7faf8813f640 2023-12-04T05:39:38.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8cc82640 1 --2- 172.21.15.111:0/3809820880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813f640 0x7faf8813fab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf8806b470 tx=0x7faf7802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c86cb2c5917b5d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf7ffff640 1 -- 172.21.15.111:0/3809820880 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780317e0 con 0x7faf8813f640 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf7ffff640 1 -- 172.21.15.111:0/3809820880 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78031940 con 0x7faf8813f640 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf7ffff640 1 -- 172.21.15.111:0/3809820880 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78031cd0 con 0x7faf8813f640 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 -- 172.21.15.111:0/3809820880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813f640 msgr2=0x7faf8813fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/3809820880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813f640 0x7faf8813fab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf8806b470 tx=0x7faf7802b880 comp rx=0 tx=0).stop 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 -- 172.21.15.111:0/3809820880 shutdown_connections 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/3809820880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813f640 0x7faf8813fab0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/3809820880 >> v2:172.21.15.192:3300/0 conn(0x7faf8813cee0 0x7faf8813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 -- 172.21.15.111:0/3809820880 >> 172.21.15.111:0/3809820880 conn(0x7faf880b0cf0 msgr2=0x7faf880b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 -- 172.21.15.111:0/3809820880 shutdown_connections 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.327+0000 7faf8f6e7640 1 -- 172.21.15.111:0/3809820880 wait complete. 2023-12-04T05:39:38.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.328+0000 7faf8f6e7640 1 Processor -- start 2023-12-04T05:39:38.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 -- start start 2023-12-04T05:39:38.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf88071d20 0x7faf881f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 0x7faf8815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf8813f640 0x7faf881f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf881f3580 con 0x7faf8813cee0 2023-12-04T05:39:38.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf88142040 con 0x7faf8813f640 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8f6e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf88141a80 con 0x7faf88071d20 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8cc82640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 0x7faf8815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8cc82640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 0x7faf8815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43572/0 (socket says 172.21.15.111:43572) 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.329+0000 7faf8cc82640 1 -- 172.21.15.111:0/2584894265 learned_addr learned my addr 172.21.15.111:0/2584894265 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8d483640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf88071d20 0x7faf881f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8dc84640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf8813f640 0x7faf881f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8cc82640 1 -- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf88071d20 msgr2=0x7faf881f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8cc82640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf88071d20 0x7faf881f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8cc82640 1 -- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf8813f640 msgr2=0x7faf881f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8cc82640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf8813f640 0x7faf881f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8cc82640 1 -- 172.21.15.111:0/2584894265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780094d0 con 0x7faf8813cee0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8dc84640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf8813f640 0x7faf881f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8cc82640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 0x7faf8815d090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faf78000c00 tx=0x7faf780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78032880 con 0x7faf8813cee0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78032a20 con 0x7faf8813cee0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78032dd0 con 0x7faf8813cee0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf88141d60 con 0x7faf8813cee0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.330+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf881f3ee0 con 0x7faf8813cee0 2023-12-04T05:39:38.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.331+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf88069360 con 0x7faf8813cee0 2023-12-04T05:39:38.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.331+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7faf7804e450 con 0x7faf8813cee0 2023-12-04T05:39:38.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.332+0000 7faf7dffb640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf6c06b2a0 0x7faf6c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.332+0000 7faf8d483640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf6c06b2a0 0x7faf6c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.332+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf780abd50 con 0x7faf8813cee0 2023-12-04T05:39:38.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.332+0000 7faf8d483640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf6c06b2a0 0x7faf6c06d760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf8806d370 tx=0x7faf8000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.337+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf780af030 con 0x7faf8813cee0 2023-12-04T05:39:38.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.502+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf88136d10 con 0x7faf8813cee0 2023-12-04T05:39:38.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.503+0000 7faf7dffb640 1 -- 172.21.15.111:0/2584894265 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf78076c80 con 0x7faf8813cee0 2023-12-04T05:39:38.504 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:38.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.504+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf6c06b2a0 msgr2=0x7faf6c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.504+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf6c06b2a0 0x7faf6c06d760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf8806d370 tx=0x7faf8000b040 comp rx=0 tx=0).stop 2023-12-04T05:39:38.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 msgr2=0x7faf8815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 0x7faf8815d090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faf78000c00 tx=0x7faf780046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 shutdown_connections 2023-12-04T05:39:38.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf6c06b2a0 0x7faf6c06d760 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf8813f640 0x7faf881f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf8813cee0 0x7faf8815d090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 --2- 172.21.15.111:0/2584894265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf88071d20 0x7faf881f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 >> 172.21.15.111:0/2584894265 conn(0x7faf880b0cf0 msgr2=0x7faf88159500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 shutdown_connections 2023-12-04T05:39:38.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.505+0000 7faf8f6e7640 1 -- 172.21.15.111:0/2584894265 wait complete. 2023-12-04T05:39:38.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:38.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.615+0000 7fc24f8ad640 1 Processor -- start 2023-12-04T05:39:38.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.615+0000 7fc24f8ad640 1 -- start start 2023-12-04T05:39:38.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24f8ad640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc24815b720 0x7fc24806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24f8ad640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2480aee60 con 0x7fc24815c060 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24f8ad640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2480cd100 con 0x7fc24806ba10 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24f8ad640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24806bec0 con 0x7fc24815b720 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24ce48640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc24815b720 0x7fc24806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24d649640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc24815c060 0x7fc2480cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51072/0 (socket says 172.21.15.111:51072) 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24d649640 1 -- 172.21.15.111:0/2145685870 learned_addr learned my addr 172.21.15.111:0/2145685870 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24ce48640 1 -- 172.21.15.111:0/2145685870 >> v1:172.21.15.192:6789/0 conn(0x7fc24806ba10 legacy=0x7fc248151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24ce48640 1 -- 172.21.15.111:0/2145685870 >> v1:172.21.15.111:6789/0 conn(0x7fc24815c060 legacy=0x7fc2480cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc24ce48640 1 -- 172.21.15.111:0/2145685870 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480cd3e0 con 0x7fc24815b720 2023-12-04T05:39:38.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.616+0000 7fc23ffff640 1 -- 172.21.15.111:0/2145685870 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553386676 0 0) 0x7fc244001490 con 0x7fc24815c060 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.617+0000 7fc24ce48640 1 --2- 172.21.15.111:0/2145685870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815b720 0x7fc24806b4d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2380097a0 tx=0x7fc23802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd8ce10481f681b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.617+0000 7fc23ffff640 1 -- 172.21.15.111:0/2145685870 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238031830 con 0x7fc24815b720 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.617+0000 7fc23ffff640 1 -- 172.21.15.111:0/2145685870 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238031990 con 0x7fc24815b720 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.617+0000 7fc23ffff640 1 -- 172.21.15.111:0/2145685870 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238031d20 con 0x7fc24815b720 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 -- 172.21.15.111:0/2145685870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815b720 msgr2=0x7fc24806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/2145685870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815b720 0x7fc24806b4d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2380097a0 tx=0x7fc23802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 -- 172.21.15.111:0/2145685870 shutdown_connections 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/2145685870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815b720 0x7fc24806b4d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 -- 172.21.15.111:0/2145685870 >> 172.21.15.111:0/2145685870 conn(0x7fc2480b1b50 msgr2=0x7fc248070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 -- 172.21.15.111:0/2145685870 shutdown_connections 2023-12-04T05:39:38.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.618+0000 7fc24f8ad640 1 -- 172.21.15.111:0/2145685870 wait complete. 2023-12-04T05:39:38.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.619+0000 7fc24f8ad640 1 Processor -- start 2023-12-04T05:39:38.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.619+0000 7fc24f8ad640 1 -- start start 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24f8ad640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc24806ba10 0x7fc248142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24f8ad640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc24815b720 0x7fc248140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24f8ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 0x7fc248141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24f8ad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2481428c0 con 0x7fc24815c060 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24f8ad640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248141cb0 con 0x7fc24815b720 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24f8ad640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc248141f90 con 0x7fc24806ba10 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 0x7fc248141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 0x7fc248141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43590/0 (socket says 172.21.15.111:43590) 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 -- 172.21.15.111:0/3381485657 learned_addr learned my addr 172.21.15.111:0/3381485657 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 -- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc24806ba10 msgr2=0x7fc248142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24d649640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc24806ba10 0x7fc248142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24ce48640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc24815b720 0x7fc248140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc24806ba10 0x7fc248142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 -- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc24815b720 msgr2=0x7fc248140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc24815b720 0x7fc248140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 -- 172.21.15.111:0/3381485657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238009400 con 0x7fc24815c060 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24d649640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc24806ba10 0x7fc248142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24ce48640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc24815b720 0x7fc248140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc24de4a640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 0x7fc248141340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c410 tx=0x7fc24000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.620+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240011790 con 0x7fc24815c060 2023-12-04T05:39:38.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.621+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2400118f0 con 0x7fc24815c060 2023-12-04T05:39:38.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.621+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc248142270 con 0x7fc24815c060 2023-12-04T05:39:38.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.621+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240011ce0 con 0x7fc24815c060 2023-12-04T05:39:38.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.621+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc248141880 con 0x7fc24815c060 2023-12-04T05:39:38.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.622+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc248141a70 con 0x7fc24815c060 2023-12-04T05:39:38.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.622+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc24001c460 con 0x7fc24815c060 2023-12-04T05:39:38.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.623+0000 7fc23dffb640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc22c06b2a0 0x7fc22c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.623+0000 7fc24d649640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc22c06b2a0 0x7fc22c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.623+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2400898c0 con 0x7fc24815c060 2023-12-04T05:39:38.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.623+0000 7fc24d649640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc22c06b2a0 0x7fc22c06d760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc244007fd0 tx=0x7fc244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.628+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc248141a70 con 0x7fc24815c060 2023-12-04T05:39:38.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.796+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc248141a70 con 0x7fc24815c060 2023-12-04T05:39:38.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.798+0000 7fc23dffb640 1 -- 172.21.15.111:0/3381485657 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fc248141a70 con 0x7fc24815c060 2023-12-04T05:39:38.799 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:38.799 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc22c06b2a0 msgr2=0x7fc22c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc22c06b2a0 0x7fc22c06d760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc244007fd0 tx=0x7fc244009040 comp rx=0 tx=0).stop 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 msgr2=0x7fc248141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 0x7fc248141340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c410 tx=0x7fc24000c8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 shutdown_connections 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc22c06b2a0 0x7fc22c06d760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc24815c060 0x7fc248141340 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc24815b720 0x7fc248140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 --2- 172.21.15.111:0/3381485657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc24806ba10 0x7fc248142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 >> 172.21.15.111:0/3381485657 conn(0x7fc2480b1b50 msgr2=0x7fc24806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 shutdown_connections 2023-12-04T05:39:38.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.800+0000 7fc24f8ad640 1 -- 172.21.15.111:0/3381485657 wait complete. 2023-12-04T05:39:38.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:38.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:38.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.914+0000 7ff3ed9ec640 1 Processor -- start 2023-12-04T05:39:38.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.914+0000 7ff3ed9ec640 1 -- start start 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3ed9ec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff3e815d480 0x7ff3e80d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3ed9ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3ed9ec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e815ddc0 con 0x7ff3e815d860 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3ed9ec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e815e0a0 con 0x7ff3e815d480 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3ed9ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e815e380 con 0x7ff3e80d1310 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3dffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3e77fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7ff3e815d860 0x7ff3e8160270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43568/0 (socket says 172.21.15.111:43568) 2023-12-04T05:39:38.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3e6ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff3e815d480 0x7ff3e80d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3e77fe640 1 -- 172.21.15.111:0/76222749 learned_addr learned my addr 172.21.15.111:0/76222749 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:38.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3dffff640 1 -- 172.21.15.111:0/76222749 >> v2:172.21.15.192:3300/0 conn(0x7ff3e815d480 msgr2=0x7ff3e80d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:38.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3dffff640 1 --2- 172.21.15.111:0/76222749 >> v2:172.21.15.192:3300/0 conn(0x7ff3e815d480 0x7ff3e80d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3dffff640 1 -- 172.21.15.111:0/76222749 >> v1:172.21.15.192:6789/0 conn(0x7ff3e815d860 legacy=0x7ff3e8160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3dffff640 1 -- 172.21.15.111:0/76222749 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e815e660 con 0x7ff3e80d1310 2023-12-04T05:39:38.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.915+0000 7ff3e67fc640 1 -- 172.21.15.111:0/76222749 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181161859 0 0) 0x7ff3e815ddc0 con 0x7ff3e815d860 2023-12-04T05:39:38.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.916+0000 7ff3dffff640 1 --2- 172.21.15.111:0/76222749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80cfad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00097a0 tx=0x7ff3d002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96a30982f938516 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.916+0000 7ff3e67fc640 1 -- 172.21.15.111:0/76222749 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0031830 con 0x7ff3e80d1310 2023-12-04T05:39:38.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.916+0000 7ff3e67fc640 1 -- 172.21.15.111:0/76222749 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d0031990 con 0x7ff3e80d1310 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.916+0000 7ff3e67fc640 1 -- 172.21.15.111:0/76222749 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0031d20 con 0x7ff3e80d1310 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/76222749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 msgr2=0x7ff3e80cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/76222749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80cfad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00097a0 tx=0x7ff3d002b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/76222749 shutdown_connections 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/76222749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80cfad0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/76222749 >> v2:172.21.15.192:3300/0 conn(0x7ff3e815d480 0x7ff3e80d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/76222749 >> 172.21.15.111:0/76222749 conn(0x7ff3e809c540 msgr2=0x7ff3e80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/76222749 shutdown_connections 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.917+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/76222749 wait complete. 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.918+0000 7ff3ed9ec640 1 Processor -- start 2023-12-04T05:39:38.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.918+0000 7ff3ed9ec640 1 -- start start 2023-12-04T05:39:38.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3ed9ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3ed9ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3e815d480 0x7ff3e80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3ed9ec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 0x7ff3e80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3ed9ec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b3af0 con 0x7ff3e815d860 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3ed9ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b07c0 con 0x7ff3e815d480 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3ed9ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b2c10 con 0x7ff3e80d1310 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 0x7ff3e80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 0x7ff3e80cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43594/0 (socket says 172.21.15.111:43594) 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 -- 172.21.15.111:0/2182462405 learned_addr learned my addr 172.21.15.111:0/2182462405 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3dffff640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3e815d480 0x7ff3e80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e6ffd640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 -- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 msgr2=0x7ff3e80b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 -- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3e815d480 msgr2=0x7ff3e80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3e815d480 0x7ff3e80b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:38.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.919+0000 7ff3e77fe640 1 -- 172.21.15.111:0/2182462405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0009400 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.920+0000 7ff3e77fe640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 0x7ff3e80cade0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000c700 tx=0x7ff3e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.920+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000ce10 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.920+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e000a640 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.920+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e80b2e70 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.920+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000aa30 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.920+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e80b3430 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.921+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8067980 con 0x7ff3e815d860 2023-12-04T05:39:38.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.921+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff3e000ab90 con 0x7ff3e815d860 2023-12-04T05:39:38.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.922+0000 7ff3e4ff9640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff3c806b2a0 0x7ff3c806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:38.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.922+0000 7ff3e6ffd640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff3c806b2a0 0x7ff3c806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:38.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.922+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3e00891b0 con 0x7ff3e815d860 2023-12-04T05:39:38.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.922+0000 7ff3e6ffd640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff3c806b2a0 0x7ff3c806d760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009b40 tx=0x7ff3d8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:38.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:38.926+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3e0054160 con 0x7ff3e815d860 2023-12-04T05:39:39.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.092+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3e80cac30 con 0x7ff3e815d860 2023-12-04T05:39:39.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.093+0000 7ff3e4ff9640 1 -- 172.21.15.111:0/2182462405 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3e0057780 con 0x7ff3e815d860 2023-12-04T05:39:39.093 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.094+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff3c806b2a0 msgr2=0x7ff3c806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.094+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff3c806b2a0 0x7ff3c806d760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009b40 tx=0x7ff3d8009190 comp rx=0 tx=0).stop 2023-12-04T05:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.094+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 msgr2=0x7ff3e80cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.094+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 0x7ff3e80cade0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000c700 tx=0x7ff3e000cbc0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 shutdown_connections 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff3c806b2a0 0x7ff3c806d760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e815d860 0x7ff3e80cade0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3e815d480 0x7ff3e80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 --2- 172.21.15.111:0/2182462405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3e80d1310 0x7ff3e80b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 >> 172.21.15.111:0/2182462405 conn(0x7ff3e809c540 msgr2=0x7ff3e815eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 shutdown_connections 2023-12-04T05:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.095+0000 7ff3ed9ec640 1 -- 172.21.15.111:0/2182462405 wait complete. 2023-12-04T05:39:39.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:39.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.211+0000 7f81cb7a5640 1 Processor -- start 2023-12-04T05:39:39.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.212+0000 7f81cb7a5640 1 -- start start 2023-12-04T05:39:39.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.212+0000 7f81cb7a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81c413cee0 0x7f81c413d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.212+0000 7f81cb7a5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f81c413f640 0x7f81c413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.212+0000 7f81cb7a5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c413d2c0 con 0x7f81c4071d20 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81cb7a5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c41400e0 con 0x7f81c413cee0 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81cb7a5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c41403c0 con 0x7f81c413f640 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81c413cee0 0x7f81c413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f81c413cee0 0x7f81c413d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43596/0 (socket says 172.21.15.111:43596) 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 -- 172.21.15.111:0/2447689390 learned_addr learned my addr 172.21.15.111:0/2447689390 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c8d40640 1 --2- 172.21.15.111:0/2447689390 >> v2:172.21.15.192:3300/0 conn(0x7f81c413f640 0x7f81c413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 -- 172.21.15.111:0/2447689390 >> v2:172.21.15.192:3300/0 conn(0x7f81c413f640 msgr2=0x7f81c413fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 --2- 172.21.15.111:0/2447689390 >> v2:172.21.15.192:3300/0 conn(0x7f81c413f640 0x7f81c413fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 -- 172.21.15.111:0/2447689390 >> v1:172.21.15.192:6789/0 conn(0x7f81c4071d20 legacy=0x7f81c4072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.213+0000 7f81c9541640 1 -- 172.21.15.111:0/2447689390 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40cdb50 con 0x7f81c413cee0 2023-12-04T05:39:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81c9541640 1 --2- 172.21.15.111:0/2447689390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c413d600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f81c4143c70 tx=0x7f81bc02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85c221cd0192b9f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81bbfff640 1 -- 172.21.15.111:0/2447689390 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc02ee00 con 0x7f81c413cee0 2023-12-04T05:39:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81bbfff640 1 -- 172.21.15.111:0/2447689390 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc036440 con 0x7f81c413cee0 2023-12-04T05:39:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81bbfff640 1 -- 172.21.15.111:0/2447689390 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0367d0 con 0x7f81c413cee0 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81cb7a5640 1 -- 172.21.15.111:0/2447689390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 msgr2=0x7f81c413d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/2447689390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c413d600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f81c4143c70 tx=0x7f81bc02e050 comp rx=0 tx=0).stop 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81cb7a5640 1 -- 172.21.15.111:0/2447689390 shutdown_connections 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.214+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/2447689390 >> v2:172.21.15.192:3300/0 conn(0x7f81c413f640 0x7f81c413fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.215+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/2447689390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c413d600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.215+0000 7f81cb7a5640 1 -- 172.21.15.111:0/2447689390 >> 172.21.15.111:0/2447689390 conn(0x7f81c40b0cf0 msgr2=0x7f81c40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.215+0000 7f81cb7a5640 1 -- 172.21.15.111:0/2447689390 shutdown_connections 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.215+0000 7f81cb7a5640 1 -- 172.21.15.111:0/2447689390 wait complete. 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 Processor -- start 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 -- start start 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81c4071d20 0x7f81c40a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81c413f640 0x7f81c415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c4143030 con 0x7f81c413cee0 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41432b0 con 0x7f81c413f640 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81cb7a5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c415c180 con 0x7f81c4071d20 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.216+0000 7f81c8d40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c415a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43608/0 (socket says 172.21.15.111:43608) 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 -- 172.21.15.111:0/3978548035 learned_addr learned my addr 172.21.15.111:0/3978548035 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c9541640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81c4071d20 0x7f81c40a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c9d42640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81c413f640 0x7f81c415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 -- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81c4071d20 msgr2=0x7f81c40a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81c4071d20 0x7f81c40a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 -- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81c413f640 msgr2=0x7f81c415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81c413f640 0x7f81c415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 -- 172.21.15.111:0/3978548035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc00b770 con 0x7f81c413cee0 2023-12-04T05:39:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c9d42640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81c413f640 0x7f81c415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81c8d40640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c415a6c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81b400e870 tx=0x7f81b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40094f0 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b400c940 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c415c460 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b400ccf0 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.217+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c40a9aa0 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.218+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c4069360 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.219+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f81b4013470 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.219+0000 7f81b9ffb640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f81a806b2a0 0x7f81a806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.219+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81b4089f00 con 0x7f81c413cee0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.219+0000 7f81c9541640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f81a806b2a0 0x7f81a806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.219+0000 7f81c9541640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f81a806b2a0 0x7f81a806d760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f81bc038000 tx=0x7f81bc032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.223+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81b4054eb0 con 0x7f81c413cee0 2023-12-04T05:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.391+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81c4070380 con 0x7f81c413cee0 2023-12-04T05:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.392+0000 7f81b9ffb640 1 -- 172.21.15.111:0/3978548035 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81b4017020 con 0x7f81c413cee0 2023-12-04T05:39:39.392 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:39.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f81a806b2a0 msgr2=0x7f81a806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f81a806b2a0 0x7f81a806d760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f81bc038000 tx=0x7f81bc032000 comp rx=0 tx=0).stop 2023-12-04T05:39:39.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 msgr2=0x7f81c415a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c415a6c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81b400e870 tx=0x7f81b400ed30 comp rx=0 tx=0).stop 2023-12-04T05:39:39.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 shutdown_connections 2023-12-04T05:39:39.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f81a806b2a0 0x7f81a806d760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81c413f640 0x7f81c415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81c413cee0 0x7f81c415a6c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 --2- 172.21.15.111:0/3978548035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81c4071d20 0x7f81c40a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 >> 172.21.15.111:0/3978548035 conn(0x7f81c40b0cf0 msgr2=0x7f81c40ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.393+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 shutdown_connections 2023-12-04T05:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.394+0000 7f81cb7a5640 1 -- 172.21.15.111:0/3978548035 wait complete. 2023-12-04T05:39:39.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:39.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.508+0000 7f736b51d640 1 Processor -- start 2023-12-04T05:39:39.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.509+0000 7f736b51d640 1 -- start start 2023-12-04T05:39:39.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.509+0000 7f736b51d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.509+0000 7f736b51d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73640d1310 0x7f73640cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.509+0000 7f736b51d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736415ddc0 con 0x7f736415d860 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.509+0000 7f736b51d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736415e0a0 con 0x7f73640d1310 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.509+0000 7f736b51d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736415e380 con 0x7f736415d480 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f7368ab8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73640d1310 0x7f73640cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f73692b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f7369aba640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f736415d860 0x7f7364160270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:43578/0 (socket says 172.21.15.111:43578) 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f7369aba640 1 -- 172.21.15.111:0/2264383811 learned_addr learned my addr 172.21.15.111:0/2264383811 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f735bfff640 1 -- 172.21.15.111:0/2264383811 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634431336 0 0) 0x7f736415ddc0 con 0x7f736415d860 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f73692b9640 1 -- 172.21.15.111:0/2264383811 >> v2:172.21.15.192:3300/0 conn(0x7f73640d1310 msgr2=0x7f73640cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f73692b9640 1 --2- 172.21.15.111:0/2264383811 >> v2:172.21.15.192:3300/0 conn(0x7f73640d1310 0x7f73640cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f73692b9640 1 -- 172.21.15.111:0/2264383811 >> v1:172.21.15.192:6789/0 conn(0x7f736415d860 legacy=0x7f7364160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.510+0000 7f73692b9640 1 -- 172.21.15.111:0/2264383811 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736415e660 con 0x7f736415d480 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f73692b9640 1 --2- 172.21.15.111:0/2264383811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640d0dd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f73600097a0 tx=0x7f736002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a2db28af3bd69b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f735bfff640 1 -- 172.21.15.111:0/2264383811 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360031830 con 0x7f736415d480 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f735bfff640 1 -- 172.21.15.111:0/2264383811 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360031990 con 0x7f736415d480 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f735bfff640 1 -- 172.21.15.111:0/2264383811 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360031d20 con 0x7f736415d480 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f736b51d640 1 -- 172.21.15.111:0/2264383811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 msgr2=0x7f73640d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f736b51d640 1 --2- 172.21.15.111:0/2264383811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640d0dd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f73600097a0 tx=0x7f736002b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f736b51d640 1 -- 172.21.15.111:0/2264383811 shutdown_connections 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f736b51d640 1 --2- 172.21.15.111:0/2264383811 >> v2:172.21.15.192:3300/0 conn(0x7f73640d1310 0x7f73640cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f736b51d640 1 --2- 172.21.15.111:0/2264383811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640d0dd0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.511+0000 7f736b51d640 1 -- 172.21.15.111:0/2264383811 >> 172.21.15.111:0/2264383811 conn(0x7f736409c540 msgr2=0x7f73640d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.512+0000 7f736b51d640 1 -- 172.21.15.111:0/2264383811 shutdown_connections 2023-12-04T05:39:39.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.512+0000 7f736b51d640 1 -- 172.21.15.111:0/2264383811 wait complete. 2023-12-04T05:39:39.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 Processor -- start 2023-12-04T05:39:39.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 -- start start 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73640d1310 0x7f736414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 0x7f73640c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73640cb380 con 0x7f736415d860 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736414c500 con 0x7f73640d1310 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.513+0000 7f736b51d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73640cb5d0 con 0x7f736415d480 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 0x7f73640c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 0x7f73640c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43616/0 (socket says 172.21.15.111:43616) 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 -- 172.21.15.111:0/3377265032 learned_addr learned my addr 172.21.15.111:0/3377265032 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7368ab8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f73692b9640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73640d1310 0x7f736414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 -- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 msgr2=0x7f73640c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 -- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73640d1310 msgr2=0x7f736414bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73640d1310 0x7f736414bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 -- 172.21.15.111:0/3377265032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009400 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7369aba640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 0x7f73640c7660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f735c00c6d0 tx=0x7f735c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c004d60 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00a640 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73640cb8b0 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00aa70 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.514+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73640cbe70 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.515+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7364067980 con 0x7f736415d860 2023-12-04T05:39:39.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.516+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f735c0070d0 con 0x7f736415d860 2023-12-04T05:39:39.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.516+0000 7f7359ffb640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f734806b2a0 0x7f734806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.516+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f735c089400 con 0x7f736415d860 2023-12-04T05:39:39.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.516+0000 7f73692b9640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f734806b2a0 0x7f734806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.516+0000 7f73692b9640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f734806b2a0 0x7f734806d760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7360036000 tx=0x7f736003e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.520+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f735c0543b0 con 0x7f736415d860 2023-12-04T05:39:39.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.685+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73640cc090 con 0x7f736415d860 2023-12-04T05:39:39.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.685+0000 7f7359ffb640 1 -- 172.21.15.111:0/3377265032 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f735c0579d0 con 0x7f736415d860 2023-12-04T05:39:39.686 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f734806b2a0 msgr2=0x7f734806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f734806b2a0 0x7f734806d760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7360036000 tx=0x7f736003e040 comp rx=0 tx=0).stop 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 msgr2=0x7f73640c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 0x7f73640c7660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f735c00c6d0 tx=0x7f735c00cb90 comp rx=0 tx=0).stop 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 shutdown_connections 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f734806b2a0 0x7f734806d760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f736415d860 0x7f73640c7660 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f736415d480 0x7f73640c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 --2- 172.21.15.111:0/3377265032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73640d1310 0x7f736414bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 >> 172.21.15.111:0/3377265032 conn(0x7f736409c540 msgr2=0x7f7364160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 shutdown_connections 2023-12-04T05:39:39.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.687+0000 7f736b51d640 1 -- 172.21.15.111:0/3377265032 wait complete. 2023-12-04T05:39:39.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:39.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.797+0000 7f73b0609640 1 Processor -- start 2023-12-04T05:39:39.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.797+0000 7f73b0609640 1 -- start start 2023-12-04T05:39:39.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73b0609640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f73a80d1780 0x7f73a80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73b0609640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a80cdcb0 con 0x7f73a80b20a0 2023-12-04T05:39:39.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73b0609640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a80d39e0 con 0x7f73a80b2ad0 2023-12-04T05:39:39.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73b0609640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a80d3cc0 con 0x7f73a80d1780 2023-12-04T05:39:39.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73ae3a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f73a80d1780 0x7f73a80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73adba4640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f73a80b20a0 0x7f73a80d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51086/0 (socket says 172.21.15.111:51086) 2023-12-04T05:39:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.798+0000 7f73adba4640 1 -- 172.21.15.111:0/787819601 learned_addr learned my addr 172.21.15.111:0/787819601 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.799+0000 7f73ae3a5640 1 -- 172.21.15.111:0/787819601 >> v1:172.21.15.192:6789/0 conn(0x7f73a80b2ad0 legacy=0x7f73a80ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.799+0000 7f73ae3a5640 1 -- 172.21.15.111:0/787819601 >> v1:172.21.15.111:6789/0 conn(0x7f73a80b20a0 legacy=0x7f73a80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.799+0000 7f73ad3a3640 1 -- 172.21.15.111:0/787819601 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099891147 0 0) 0x7f73a80cdcb0 con 0x7f73a80b20a0 2023-12-04T05:39:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.799+0000 7f73ae3a5640 1 -- 172.21.15.111:0/787819601 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80cdf50 con 0x7f73a80d1780 2023-12-04T05:39:39.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.799+0000 7f73ae3a5640 1 --2- 172.21.15.111:0/787819601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80d1780 0x7f73a80d01e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f73980097a0 tx=0x7f739802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc065e14bbff1131 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73ad3a3640 1 -- 172.21.15.111:0/787819601 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739802e070 con 0x7f73a80d1780 2023-12-04T05:39:39.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73ad3a3640 1 -- 172.21.15.111:0/787819601 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980315a0 con 0x7f73a80d1780 2023-12-04T05:39:39.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73ad3a3640 1 -- 172.21.15.111:0/787819601 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398031930 con 0x7f73a80d1780 2023-12-04T05:39:39.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 -- 172.21.15.111:0/787819601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80d1780 msgr2=0x7f73a80d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 --2- 172.21.15.111:0/787819601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80d1780 0x7f73a80d01e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f73980097a0 tx=0x7f739802b840 comp rx=0 tx=0).stop 2023-12-04T05:39:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 -- 172.21.15.111:0/787819601 shutdown_connections 2023-12-04T05:39:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 --2- 172.21.15.111:0/787819601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80d1780 0x7f73a80d01e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 -- 172.21.15.111:0/787819601 >> 172.21.15.111:0/787819601 conn(0x7f73a809c4e0 msgr2=0x7f73a80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 -- 172.21.15.111:0/787819601 shutdown_connections 2023-12-04T05:39:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.800+0000 7f73b0609640 1 -- 172.21.15.111:0/787819601 wait complete. 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.801+0000 7f73b0609640 1 Processor -- start 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.801+0000 7f73b0609640 1 -- start start 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73b0609640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73a80b20a0 0x7f73a8160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73b0609640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 0x7f73a815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73b0609640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73a80d1780 0x7f73a80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73b0609640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a815ee30 con 0x7f73a80b2ad0 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73b0609640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a815dfc0 con 0x7f73a80d1780 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73b0609640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a815e220 con 0x7f73a80b20a0 2023-12-04T05:39:39.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73adba4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 0x7f73a815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73adba4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 0x7f73a815da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43636/0 (socket says 172.21.15.111:43636) 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.802+0000 7f73adba4640 1 -- 172.21.15.111:0/1480823549 learned_addr learned my addr 172.21.15.111:0/1480823549 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73adba4640 1 -- 172.21.15.111:0/1480823549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73a80b20a0 msgr2=0x7f73a8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73adba4640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73a80b20a0 0x7f73a8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73adba4640 1 -- 172.21.15.111:0/1480823549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73a80d1780 msgr2=0x7f73a80b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73adba4640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73a80d1780 0x7f73a80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73adba4640 1 -- 172.21.15.111:0/1480823549 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7398009400 con 0x7f73a80b2ad0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73adba4640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 0x7f73a815da80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f73a400c4d0 tx=0x7f73a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a40074e0 con 0x7f73a80b2ad0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a4007660 con 0x7f73a80b2ad0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a815e500 con 0x7f73a80b2ad0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4004a20 con 0x7f73a80b2ad0 2023-12-04T05:39:39.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.803+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a815eac0 con 0x7f73a80b2ad0 2023-12-04T05:39:39.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.804+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a815ec80 con 0x7f73a80b2ad0 2023-12-04T05:39:39.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.804+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f73a4004b80 con 0x7f73a80b2ad0 2023-12-04T05:39:39.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.805+0000 7f739effd640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2a0 0x7f738c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:39.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.805+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73a4089080 con 0x7f73a80b2ad0 2023-12-04T05:39:39.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.805+0000 7f73ae3a5640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2a0 0x7f738c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:39.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.805+0000 7f73ae3a5640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2a0 0x7f738c06d760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7398036000 tx=0x7f7398030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:39.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.809+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73a4053f80 con 0x7f73a80b2ad0 2023-12-04T05:39:39.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.976+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73a80693d0 con 0x7f73a80b2ad0 2023-12-04T05:39:39.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.977+0000 7f739effd640 1 -- 172.21.15.111:0/1480823549 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f73a40575a0 con 0x7f73a80b2ad0 2023-12-04T05:39:39.978 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:39.978 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2a0 msgr2=0x7f738c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2a0 0x7f738c06d760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7398036000 tx=0x7f7398030000 comp rx=0 tx=0).stop 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 msgr2=0x7f73a815da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 0x7f73a815da80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f73a400c4d0 tx=0x7f73a400c990 comp rx=0 tx=0).stop 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 shutdown_connections 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2a0 0x7f738c06d760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73a80d1780 0x7f73a80b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73a80b2ad0 0x7f73a815da80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 --2- 172.21.15.111:0/1480823549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73a80b20a0 0x7f73a8160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:39.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 >> 172.21.15.111:0/1480823549 conn(0x7f73a809c4e0 msgr2=0x7f73a80d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:39.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 shutdown_connections 2023-12-04T05:39:39.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:39.979+0000 7f73b0609640 1 -- 172.21.15.111:0/1480823549 wait complete. 2023-12-04T05:39:39.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:39.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:40.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.094+0000 7ff9b7d3c640 1 Processor -- start 2023-12-04T05:39:40.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.094+0000 7ff9b7d3c640 1 -- start start 2023-12-04T05:39:40.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b7d3c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9b013cee0 0x7ff9b013d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b7d3c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff9b013f640 0x7ff9b013fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b7d3c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b013d2c0 con 0x7ff9b0071d20 2023-12-04T05:39:40.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b7d3c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b01400e0 con 0x7ff9b013cee0 2023-12-04T05:39:40.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b7d3c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b01403c0 con 0x7ff9b013f640 2023-12-04T05:39:40.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b5ad8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9b013cee0 0x7ff9b013d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b62d9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff9b0071d20 0x7ff9b0072130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:51096/0 (socket says 172.21.15.111:51096) 2023-12-04T05:39:40.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.095+0000 7ff9b62d9640 1 -- 172.21.15.111:0/494661852 learned_addr learned my addr 172.21.15.111:0/494661852 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b5ad8640 1 -- 172.21.15.111:0/494661852 >> v2:172.21.15.192:3300/0 conn(0x7ff9b013f640 msgr2=0x7ff9b013fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b52d7640 1 --2- 172.21.15.111:0/494661852 >> v2:172.21.15.192:3300/0 conn(0x7ff9b013f640 0x7ff9b013fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b5ad8640 1 --2- 172.21.15.111:0/494661852 >> v2:172.21.15.192:3300/0 conn(0x7ff9b013f640 0x7ff9b013fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b5ad8640 1 -- 172.21.15.111:0/494661852 >> v1:172.21.15.111:6789/0 conn(0x7ff9b0071d20 legacy=0x7ff9b0072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b4ad6640 1 -- 172.21.15.111:0/494661852 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114020367 0 0) 0x7ff9b013d2c0 con 0x7ff9b0071d20 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b5ad8640 1 -- 172.21.15.111:0/494661852 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00cdb50 con 0x7ff9b013cee0 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b5ad8640 1 --2- 172.21.15.111:0/494661852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b013d600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0143c70 tx=0x7ff9a002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e3078d208113b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b4ad6640 1 -- 172.21.15.111:0/494661852 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a002cb60 con 0x7ff9b013cee0 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b4ad6640 1 -- 172.21.15.111:0/494661852 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a002ccc0 con 0x7ff9b013cee0 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.096+0000 7ff9b4ad6640 1 -- 172.21.15.111:0/494661852 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0034660 con 0x7ff9b013cee0 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/494661852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 msgr2=0x7ff9b013d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/494661852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b013d600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0143c70 tx=0x7ff9a002bdb0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/494661852 shutdown_connections 2023-12-04T05:39:40.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/494661852 >> v2:172.21.15.192:3300/0 conn(0x7ff9b013f640 0x7ff9b013fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/494661852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b013d600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/494661852 >> 172.21.15.111:0/494661852 conn(0x7ff9b00b0cf0 msgr2=0x7ff9b00b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/494661852 shutdown_connections 2023-12-04T05:39:40.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.097+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/494661852 wait complete. 2023-12-04T05:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.098+0000 7ff9b7d3c640 1 Processor -- start 2023-12-04T05:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.098+0000 7ff9b7d3c640 1 -- start start 2023-12-04T05:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b7d3c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9b0071d20 0x7ff9b0151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b7d3c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b00ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b7d3c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9b013f640 0x7ff9b0131270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b7d3c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0153950 con 0x7ff9b013cee0 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b7d3c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b01317b0 con 0x7ff9b0071d20 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b7d3c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b014ea30 con 0x7ff9b013f640 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b00ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b00ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43656/0 (socket says 172.21.15.111:43656) 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 -- 172.21.15.111:0/896044489 learned_addr learned my addr 172.21.15.111:0/896044489 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 -- 172.21.15.111:0/896044489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9b013f640 msgr2=0x7ff9b0131270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9b013f640 0x7ff9b0131270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 -- 172.21.15.111:0/896044489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9b0071d20 msgr2=0x7ff9b0151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9b0071d20 0x7ff9b0151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 -- 172.21.15.111:0/896044489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0009a70 con 0x7ff9b013cee0 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.099+0000 7ff9b52d7640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b00ade40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c7f0 tx=0x7ff9a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.100+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4004220 con 0x7ff9b013cee0 2023-12-04T05:39:40.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.100+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a40043a0 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.100+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4007860 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.100+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b01535f0 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.100+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b0131ca0 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.101+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b0153a90 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.101+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff9a4007bd0 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.101+0000 7ff9ae7fc640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff99c06b2a0 0x7ff99c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.101+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9a4089b50 con 0x7ff9b013cee0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.101+0000 7ff9b5ad8640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff99c06b2a0 0x7ff99c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.102+0000 7ff9b5ad8640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff99c06b2a0 0x7ff99c06d760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0036000 tx=0x7ff9a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.106+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9b0153a90 con 0x7ff9b013cee0 2023-12-04T05:39:40.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.275+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9b0153a90 con 0x7ff9b013cee0 2023-12-04T05:39:40.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.277+0000 7ff9ae7fc640 1 -- 172.21.15.111:0/896044489 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9b0153a90 con 0x7ff9b013cee0 2023-12-04T05:39:40.277 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff99c06b2a0 msgr2=0x7ff99c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff99c06b2a0 0x7ff99c06d760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0036000 tx=0x7ff9a002e000 comp rx=0 tx=0).stop 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 msgr2=0x7ff9b00ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b00ade40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c7f0 tx=0x7ff9a400ccb0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 shutdown_connections 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff99c06b2a0 0x7ff99c06d760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9b013f640 0x7ff9b0131270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9b013cee0 0x7ff9b00ade40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 --2- 172.21.15.111:0/896044489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9b0071d20 0x7ff9b0151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 >> 172.21.15.111:0/896044489 conn(0x7ff9b00b0cf0 msgr2=0x7ff9b00aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 shutdown_connections 2023-12-04T05:39:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.279+0000 7ff9b7d3c640 1 -- 172.21.15.111:0/896044489 wait complete. 2023-12-04T05:39:40.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:40.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.391+0000 7fe2332c9640 1 Processor -- start 2023-12-04T05:39:40.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.391+0000 7fe2332c9640 1 -- start start 2023-12-04T05:39:40.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe2332c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe22c13cee0 0x7fe22c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe2332c9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe22c13f640 0x7fe22c13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe2332c9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c13d2c0 con 0x7fe22c071d20 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe2332c9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22c1400e0 con 0x7fe22c13f640 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe2332c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22c1403c0 con 0x7fe22c13cee0 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe22c13f640 0x7fe22c13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe22c13f640 0x7fe22c13fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43664/0 (socket says 172.21.15.111:43664) 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 -- 172.21.15.111:0/2472082334 learned_addr learned my addr 172.21.15.111:0/2472082334 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 -- 172.21.15.111:0/2472082334 >> v2:172.21.15.192:3300/0 conn(0x7fe22c13cee0 msgr2=0x7fe22c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 --2- 172.21.15.111:0/2472082334 >> v2:172.21.15.192:3300/0 conn(0x7fe22c13cee0 0x7fe22c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 -- 172.21.15.111:0/2472082334 >> v1:172.21.15.192:6789/0 conn(0x7fe22c071d20 legacy=0x7fe22c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:40.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.392+0000 7fe230864640 1 -- 172.21.15.111:0/2472082334 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0cdb50 con 0x7fe22c13f640 2023-12-04T05:39:40.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.393+0000 7fe230864640 1 --2- 172.21.15.111:0/2472082334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13f640 0x7fe22c13fab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe22c06b470 tx=0x7fe21c02da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3420f527efea92 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.393+0000 7fe223fff640 1 -- 172.21.15.111:0/2472082334 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c031820 con 0x7fe22c13f640 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.393+0000 7fe223fff640 1 -- 172.21.15.111:0/2472082334 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c031980 con 0x7fe22c13f640 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.393+0000 7fe223fff640 1 -- 172.21.15.111:0/2472082334 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c031d10 con 0x7fe22c13f640 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 -- 172.21.15.111:0/2472082334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13f640 msgr2=0x7fe22c13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 --2- 172.21.15.111:0/2472082334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13f640 0x7fe22c13fab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe22c06b470 tx=0x7fe21c02da00 comp rx=0 tx=0).stop 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 -- 172.21.15.111:0/2472082334 shutdown_connections 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 --2- 172.21.15.111:0/2472082334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13f640 0x7fe22c13fab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 --2- 172.21.15.111:0/2472082334 >> v2:172.21.15.192:3300/0 conn(0x7fe22c13cee0 0x7fe22c13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 -- 172.21.15.111:0/2472082334 >> 172.21.15.111:0/2472082334 conn(0x7fe22c0b0cf0 msgr2=0x7fe22c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 -- 172.21.15.111:0/2472082334 shutdown_connections 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.394+0000 7fe2332c9640 1 -- 172.21.15.111:0/2472082334 wait complete. 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.395+0000 7fe2332c9640 1 Processor -- start 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.395+0000 7fe2332c9640 1 -- start start 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe22c071d20 0x7fe22c136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 0x7fe22c0adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe22c13f640 0x7fe22c13b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c136d70 con 0x7fe22c13cee0 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1e34b0 con 0x7fe22c13f640 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1e3750 con 0x7fe22c071d20 2023-12-04T05:39:40.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 0x7fe22c0adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 0x7fe22c0adb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43678/0 (socket says 172.21.15.111:43678) 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 -- 172.21.15.111:0/923154198 learned_addr learned my addr 172.21.15.111:0/923154198 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe231065640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe22c071d20 0x7fe22c136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe231866640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe22c13f640 0x7fe22c13b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 -- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe22c071d20 msgr2=0x7fe22c136830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe22c071d20 0x7fe22c136830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 -- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe22c13f640 msgr2=0x7fe22c13b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe22c13f640 0x7fe22c13b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 -- 172.21.15.111:0/923154198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c0094d0 con 0x7fe22c13cee0 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe231065640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe22c071d20 0x7fe22c136830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe230864640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 0x7fe22c0adb60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe21c000c00 tx=0x7fe21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c039bc0 con 0x7fe22c13cee0 2023-12-04T05:39:40.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c039d20 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c1e3a30 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.396+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c043630 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.397+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c1e3ff0 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.398+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c069360 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.398+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe21c02f040 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.398+0000 7fe221ffb640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe21006b2a0 0x7fe21006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.398+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe21c0abd70 con 0x7fe22c13cee0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.399+0000 7fe231065640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe21006b2a0 0x7fe21006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.399+0000 7fe231065640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe21006b2a0 0x7fe21006d760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe224009fa0 tx=0x7fe224009570 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.403+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe21c076be0 con 0x7fe22c13cee0 2023-12-04T05:39:40.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.568+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe22c070380 con 0x7fe22c13cee0 2023-12-04T05:39:40.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.569+0000 7fe221ffb640 1 -- 172.21.15.111:0/923154198 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe21c07a200 con 0x7fe22c13cee0 2023-12-04T05:39:40.569 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:40.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe21006b2a0 msgr2=0x7fe21006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe21006b2a0 0x7fe21006d760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe224009fa0 tx=0x7fe224009570 comp rx=0 tx=0).stop 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 msgr2=0x7fe22c0adb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 0x7fe22c0adb60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe21c000c00 tx=0x7fe21c0046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 shutdown_connections 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe21006b2a0 0x7fe21006d760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe22c13f640 0x7fe22c13b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe22c13cee0 0x7fe22c0adb60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 --2- 172.21.15.111:0/923154198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe22c071d20 0x7fe22c136830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 >> 172.21.15.111:0/923154198 conn(0x7fe22c0b0cf0 msgr2=0x7fe22c070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 shutdown_connections 2023-12-04T05:39:40.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.571+0000 7fe2332c9640 1 -- 172.21.15.111:0/923154198 wait complete. 2023-12-04T05:39:40.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:40.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.683+0000 7f0f6021c640 1 Processor -- start 2023-12-04T05:39:40.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 -- start start 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0f58070c10 0x7f0f580ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f580b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0f58070ff0 0x7f0f58071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f58159c20 con 0x7f0f58070ff0 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580ced70 con 0x7f0f58070c10 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.684+0000 7f0f6021c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580b50e0 con 0x7f0f580cd020 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0f58070ff0 0x7f0f58071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0f58070ff0 0x7f0f58071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36478/0 (socket says 172.21.15.111:36478) 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 -- 172.21.15.111:0/1613455594 learned_addr learned my addr 172.21.15.111:0/1613455594 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5d7b7640 1 --2- 172.21.15.111:0/1613455594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f580b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5dfb8640 1 --2- 172.21.15.111:0/1613455594 >> v2:172.21.15.192:3300/0 conn(0x7f0f58070c10 0x7f0f580ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 -- 172.21.15.111:0/1613455594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 msgr2=0x7f0f580b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:40.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 --2- 172.21.15.111:0/1613455594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f580b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 -- 172.21.15.111:0/1613455594 >> v2:172.21.15.192:3300/0 conn(0x7f0f58070c10 msgr2=0x7f0f580ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:40.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 --2- 172.21.15.111:0/1613455594 >> v2:172.21.15.192:3300/0 conn(0x7f0f58070c10 0x7f0f580ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.685+0000 7f0f5e7b9640 1 -- 172.21.15.111:0/1613455594 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580cd790 con 0x7f0f58070ff0 2023-12-04T05:39:40.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f5e7b9640 1 --2- 172.21.15.111:0/1613455594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f58071460 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f5806bf00 tx=0x7f0f5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3885477a0a1ad48a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f5cfb6640 1 -- 172.21.15.111:0/1613455594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5002ca70 con 0x7f0f58070ff0 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f5cfb6640 1 -- 172.21.15.111:0/1613455594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5002cbd0 con 0x7f0f58070ff0 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f5cfb6640 1 -- 172.21.15.111:0/1613455594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500343f0 con 0x7f0f58070ff0 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 -- 172.21.15.111:0/1613455594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 msgr2=0x7f0f58071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1613455594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f58071460 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f5806bf00 tx=0x7f0f5002bed0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 -- 172.21.15.111:0/1613455594 shutdown_connections 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1613455594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f58071460 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1613455594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f580b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1613455594 >> v2:172.21.15.192:3300/0 conn(0x7f0f58070c10 0x7f0f580ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 -- 172.21.15.111:0/1613455594 >> 172.21.15.111:0/1613455594 conn(0x7f0f580b0cf0 msgr2=0x7f0f580b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 -- 172.21.15.111:0/1613455594 shutdown_connections 2023-12-04T05:39:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.686+0000 7f0f6021c640 1 -- 172.21.15.111:0/1613455594 wait complete. 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 Processor -- start 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 -- start start 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f58070c10 0x7f0f58136060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f581535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f5813c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58153b30 con 0x7f0f58070ff0 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58134fc0 con 0x7f0f58070c10 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f6021c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581352a0 con 0x7f0f580cd020 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f5d7b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f581535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f5d7b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f581535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36494/0 (socket says 172.21.15.111:36494) 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.688+0000 7f0f5d7b7640 1 -- 172.21.15.111:0/1867308837 learned_addr learned my addr 172.21.15.111:0/1867308837 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5e7b9640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f5813c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5d7b7640 1 -- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 msgr2=0x7f0f5813c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5dfb8640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f58070c10 0x7f0f58136060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5d7b7640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f5813c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5d7b7640 1 -- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f58070c10 msgr2=0x7f0f58136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5d7b7640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f58070c10 0x7f0f58136060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5d7b7640 1 -- 172.21.15.111:0/1867308837 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50009b30 con 0x7f0f58070ff0 2023-12-04T05:39:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f5d7b7640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f581535f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e870 tx=0x7f0f5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540094f0 con 0x7f0f58070ff0 2023-12-04T05:39:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5400c940 con 0x7f0f58070ff0 2023-12-04T05:39:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400ccf0 con 0x7f0f58070ff0 2023-12-04T05:39:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f58135580 con 0x7f0f58070ff0 2023-12-04T05:39:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.689+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f58135b40 con 0x7f0f58070ff0 2023-12-04T05:39:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.690+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f58069360 con 0x7f0f58070ff0 2023-12-04T05:39:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.690+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0f54013470 con 0x7f0f58070ff0 2023-12-04T05:39:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.691+0000 7f0f4e7fc640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0f3c06b2a0 0x7f0f3c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.691+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f54089b80 con 0x7f0f58070ff0 2023-12-04T05:39:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.691+0000 7f0f5dfb8640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0f3c06b2a0 0x7f0f3c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.691+0000 7f0f5dfb8640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0f3c06b2a0 0x7f0f3c06d760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009a10 tx=0x7f0f48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.695+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f54054a80 con 0x7f0f58070ff0 2023-12-04T05:39:40.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.867+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f58154f50 con 0x7f0f58070ff0 2023-12-04T05:39:40.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.867+0000 7f0f4e7fc640 1 -- 172.21.15.111:0/1867308837 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f540170a0 con 0x7f0f58070ff0 2023-12-04T05:39:40.868 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:40.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0f3c06b2a0 msgr2=0x7f0f3c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0f3c06b2a0 0x7f0f3c06d760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009a10 tx=0x7f0f48008040 comp rx=0 tx=0).stop 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 msgr2=0x7f0f581535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f581535f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e870 tx=0x7f0f5400ed30 comp rx=0 tx=0).stop 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 shutdown_connections 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0f3c06b2a0 0x7f0f3c06d760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f580cd020 0x7f0f5813c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58070ff0 0x7f0f581535f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 --2- 172.21.15.111:0/1867308837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f58070c10 0x7f0f58136060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 >> 172.21.15.111:0/1867308837 conn(0x7f0f580b0cf0 msgr2=0x7f0f580af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 shutdown_connections 2023-12-04T05:39:40.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.869+0000 7f0f6021c640 1 -- 172.21.15.111:0/1867308837 wait complete. 2023-12-04T05:39:40.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:40.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.979+0000 7f3b5cc75640 1 Processor -- start 2023-12-04T05:39:40.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.979+0000 7f3b5cc75640 1 -- start start 2023-12-04T05:39:40.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.980+0000 7f3b5cc75640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b58146a20 0x7f3b580b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.980+0000 7f3b5cc75640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 0x7f3b5806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.980+0000 7f3b5cc75640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5815db90 con 0x7f3b58071500 2023-12-04T05:39:40.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.980+0000 7f3b5cc75640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5806ae70 con 0x7f3b58146a20 2023-12-04T05:39:40.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.980+0000 7f3b5cc75640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580b3650 con 0x7f3b5806a4c0 2023-12-04T05:39:40.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 0x7f3b5806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b5659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b58146a20 0x7f3b580b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 0x7f3b5806a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:55940/0 (socket says 172.21.15.111:55940) 2023-12-04T05:39:40.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 -- 172.21.15.111:0/1237024033 learned_addr learned my addr 172.21.15.111:0/1237024033 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 -- 172.21.15.111:0/1237024033 >> v2:172.21.15.192:3300/0 conn(0x7f3b58146a20 msgr2=0x7f3b580b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:40.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 --2- 172.21.15.111:0/1237024033 >> v2:172.21.15.192:3300/0 conn(0x7f3b58146a20 0x7f3b580b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 -- 172.21.15.111:0/1237024033 >> v1:172.21.15.192:6789/0 conn(0x7f3b58071500 legacy=0x7f3b5815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.981+0000 7f3b56d9d640 1 -- 172.21.15.111:0/1237024033 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5815d840 con 0x7f3b5806a4c0 2023-12-04T05:39:40.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b56d9d640 1 --2- 172.21.15.111:0/1237024033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 0x7f3b5806a930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009870 tx=0x7f3b5002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e9dc66f7632459 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5559a640 1 -- 172.21.15.111:0/1237024033 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50031640 con 0x7f3b5806a4c0 2023-12-04T05:39:40.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5559a640 1 -- 172.21.15.111:0/1237024033 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b500317a0 con 0x7f3b5806a4c0 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5559a640 1 -- 172.21.15.111:0/1237024033 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50031b30 con 0x7f3b5806a4c0 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 -- 172.21.15.111:0/1237024033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 msgr2=0x7f3b5806a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/1237024033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 0x7f3b5806a930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009870 tx=0x7f3b5002b8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 -- 172.21.15.111:0/1237024033 shutdown_connections 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/1237024033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b5806a4c0 0x7f3b5806a930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/1237024033 >> v2:172.21.15.192:3300/0 conn(0x7f3b58146a20 0x7f3b580b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 -- 172.21.15.111:0/1237024033 >> 172.21.15.111:0/1237024033 conn(0x7f3b5806b5d0 msgr2=0x7f3b5806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.982+0000 7f3b5cc75640 1 -- 172.21.15.111:0/1237024033 shutdown_connections 2023-12-04T05:39:40.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.983+0000 7f3b5cc75640 1 -- 172.21.15.111:0/1237024033 wait complete. 2023-12-04T05:39:40.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 Processor -- start 2023-12-04T05:39:40.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 -- start start 2023-12-04T05:39:40.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 0x7f3b58140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b58071500 0x7f3b5815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b58146a20 0x7f3b581f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58136000 con 0x7f3b5806a4c0 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58143ed0 con 0x7f3b58071500 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.984+0000 7f3b5cc75640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58140a10 con 0x7f3b58146a20 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 0x7f3b58140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 0x7f3b58140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36510/0 (socket says 172.21.15.111:36510) 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 -- 172.21.15.111:0/2614596809 learned_addr learned my addr 172.21.15.111:0/2614596809 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b55d9b640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b58071500 0x7f3b5815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b56d9d640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b58146a20 0x7f3b581f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 -- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b58146a20 msgr2=0x7f3b581f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b58146a20 0x7f3b581f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 -- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b58071500 msgr2=0x7f3b5815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b58071500 0x7f3b5815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:40.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 -- 172.21.15.111:0/2614596809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500094d0 con 0x7f3b5806a4c0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5659c640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 0x7f3b58140350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800efc0 tx=0x7f3b4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48014480 con 0x7f3b5806a4c0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4800f040 con 0x7f3b5806a4c0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b58133090 con 0x7f3b5806a4c0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48014800 con 0x7f3b5806a4c0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.985+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b58133650 con 0x7f3b5806a4c0 2023-12-04T05:39:40.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.986+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b580693d0 con 0x7f3b5806a4c0 2023-12-04T05:39:40.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.987+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3b48014960 con 0x7f3b5806a4c0 2023-12-04T05:39:40.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.987+0000 7f3b46ffd640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b3806b2a0 0x7f3b3806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:40.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.987+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b4808a600 con 0x7f3b5806a4c0 2023-12-04T05:39:40.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.987+0000 7f3b55d9b640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b3806b2a0 0x7f3b3806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:40.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.987+0000 7f3b55d9b640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b3806b2a0 0x7f3b3806d760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b580cf750 tx=0x7f3b4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:40.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:40.991+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b480555b0 con 0x7f3b5806a4c0 2023-12-04T05:39:41.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.161+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b5806d920 con 0x7f3b5806a4c0 2023-12-04T05:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.162+0000 7f3b46ffd640 1 -- 172.21.15.111:0/2614596809 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f3b48058bd0 con 0x7f3b5806a4c0 2023-12-04T05:39:41.167 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:41.167 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:41.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b3806b2a0 msgr2=0x7f3b3806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b3806b2a0 0x7f3b3806d760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b580cf750 tx=0x7f3b4c009040 comp rx=0 tx=0).stop 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 msgr2=0x7f3b58140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 0x7f3b58140350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800efc0 tx=0x7f3b4800c490 comp rx=0 tx=0).stop 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 shutdown_connections 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b3806b2a0 0x7f3b3806d760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b58146a20 0x7f3b581f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b58071500 0x7f3b5815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 --2- 172.21.15.111:0/2614596809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b5806a4c0 0x7f3b58140350 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 >> 172.21.15.111:0/2614596809 conn(0x7f3b5806b5d0 msgr2=0x7f3b58071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 shutdown_connections 2023-12-04T05:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.164+0000 7f3b5cc75640 1 -- 172.21.15.111:0/2614596809 wait complete. 2023-12-04T05:39:41.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:41.176 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:41.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.281+0000 7ff2cbaec640 1 Processor -- start 2023-12-04T05:39:41.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.281+0000 7ff2cbaec640 1 -- start start 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2cbaec640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2c4146b10 0x7ff2c4161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2cbaec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c415dfb0 0x7ff2c40d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2cbaec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c406eaf0 con 0x7ff2c40d0f10 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2cbaec640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c40d18e0 con 0x7ff2c4146b10 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2cbaec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c415e4e0 con 0x7ff2c415dfb0 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2c4146b10 0x7ff2c4161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff2c4146b10 0x7ff2c4161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36524/0 (socket says 172.21.15.111:36524) 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 -- 172.21.15.111:0/131652997 learned_addr learned my addr 172.21.15.111:0/131652997 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2ca089640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c415dfb0 0x7ff2c40d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 -- 172.21.15.111:0/131652997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c415dfb0 msgr2=0x7ff2c40d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:41.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c415dfb0 0x7ff2c40d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 -- 172.21.15.111:0/131652997 >> v1:172.21.15.192:6789/0 conn(0x7ff2c40d0f10 legacy=0x7ff2c415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2c9888640 1 -- 172.21.15.111:0/131652997 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c415e7c0 con 0x7ff2c4146b10 2023-12-04T05:39:41.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.282+0000 7ff2ca089640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c415dfb0 0x7ff2c40d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.283+0000 7ff2c9888640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c4146b10 0x7ff2c4161a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009870 tx=0x7ff2bc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a6529a325bb9bb server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2c8886640 1 -- 172.21.15.111:0/131652997 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc030840 con 0x7ff2c4146b10 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2c8886640 1 -- 172.21.15.111:0/131652997 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc0309a0 con 0x7ff2c4146b10 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2c8886640 1 -- 172.21.15.111:0/131652997 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc030d30 con 0x7ff2c4146b10 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 -- 172.21.15.111:0/131652997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c4146b10 msgr2=0x7ff2c4161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c4146b10 0x7ff2c4161a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009870 tx=0x7ff2bc02b980 comp rx=0 tx=0).stop 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 -- 172.21.15.111:0/131652997 shutdown_connections 2023-12-04T05:39:41.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c415dfb0 0x7ff2c40d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/131652997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c4146b10 0x7ff2c4161a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 -- 172.21.15.111:0/131652997 >> 172.21.15.111:0/131652997 conn(0x7ff2c4070af0 msgr2=0x7ff2c40662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:41.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 -- 172.21.15.111:0/131652997 shutdown_connections 2023-12-04T05:39:41.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.284+0000 7ff2cbaec640 1 -- 172.21.15.111:0/131652997 wait complete. 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 Processor -- start 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 -- start start 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c40d0f10 0x7ff2c414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2c4146b10 0x7ff2c41e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 0x7ff2c413dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4141d00 con 0x7ff2c415dfb0 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4141fe0 con 0x7ff2c4146b10 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2cbaec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41422c0 con 0x7ff2c40d0f10 2023-12-04T05:39:41.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2ca089640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 0x7ff2c413dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2ca089640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 0x7ff2c413dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36540/0 (socket says 172.21.15.111:36540) 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.286+0000 7ff2ca089640 1 -- 172.21.15.111:0/2038629374 learned_addr learned my addr 172.21.15.111:0/2038629374 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2c9888640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c40d0f10 0x7ff2c414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2c9087640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2c4146b10 0x7ff2c41e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2ca089640 1 -- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c40d0f10 msgr2=0x7ff2c414d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2ca089640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c40d0f10 0x7ff2c414d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2ca089640 1 -- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2c4146b10 msgr2=0x7ff2c41e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2ca089640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2c4146b10 0x7ff2c41e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2ca089640 1 -- 172.21.15.111:0/2038629374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc0094d0 con 0x7ff2c415dfb0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2ca089640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 0x7ff2c413dc20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4007ef0 tx=0x7ff2b400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4003b90 con 0x7ff2c415dfb0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b400f040 con 0x7ff2c415dfb0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c41425a0 con 0x7ff2c415dfb0 2023-12-04T05:39:41.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400de80 con 0x7ff2c415dfb0 2023-12-04T05:39:41.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.287+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c41e9810 con 0x7ff2c415dfb0 2023-12-04T05:39:41.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.288+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c4142ac0 con 0x7ff2c415dfb0 2023-12-04T05:39:41.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.288+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff2b4004030 con 0x7ff2c415dfb0 2023-12-04T05:39:41.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.289+0000 7ff2c27fc640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff2b006b2a0 0x7ff2b006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.289+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2b4089e00 con 0x7ff2c415dfb0 2023-12-04T05:39:41.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.289+0000 7ff2c9888640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff2b006b2a0 0x7ff2b006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.289+0000 7ff2c9888640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff2b006b2a0 0x7ff2b006d760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc036000 tx=0x7ff2bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.293+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2b4054d20 con 0x7ff2c415dfb0 2023-12-04T05:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.464+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2c4069360 con 0x7ff2c415dfb0 2023-12-04T05:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.465+0000 7ff2c27fc640 1 -- 172.21.15.111:0/2038629374 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2b401a0a0 con 0x7ff2c415dfb0 2023-12-04T05:39:41.465 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff2b006b2a0 msgr2=0x7ff2b006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff2b006b2a0 0x7ff2b006d760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc036000 tx=0x7ff2bc008030 comp rx=0 tx=0).stop 2023-12-04T05:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 msgr2=0x7ff2c413dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 0x7ff2c413dc20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4007ef0 tx=0x7ff2b400d4a0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 shutdown_connections 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff2b006b2a0 0x7ff2b006d760 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2c415dfb0 0x7ff2c413dc20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2c4146b10 0x7ff2c41e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 --2- 172.21.15.111:0/2038629374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2c40d0f10 0x7ff2c414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 >> 172.21.15.111:0/2038629374 conn(0x7ff2c4070af0 msgr2=0x7ff2c415f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 shutdown_connections 2023-12-04T05:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.467+0000 7ff2cbaec640 1 -- 172.21.15.111:0/2038629374 wait complete. 2023-12-04T05:39:41.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:41.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.579+0000 7f70b39f5640 1 Processor -- start 2023-12-04T05:39:41.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.579+0000 7f70b39f5640 1 -- start start 2023-12-04T05:39:41.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b39f5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70ac146b10 0x7f70ac161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b39f5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b39f5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ac06eaf0 con 0x7f70ac0d0f10 2023-12-04T05:39:41.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b39f5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac0d18e0 con 0x7f70ac146b10 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b39f5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15e4e0 con 0x7f70ac15dfb0 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70ac146b10 0x7f70ac161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70ac146b10 0x7f70ac161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36546/0 (socket says 172.21.15.111:36546) 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 -- 172.21.15.111:0/351255536 learned_addr learned my addr 172.21.15.111:0/351255536 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1f92640 1 --2- 172.21.15.111:0/351255536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 -- 172.21.15.111:0/351255536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 msgr2=0x7f70ac0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 --2- 172.21.15.111:0/351255536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac0d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 -- 172.21.15.111:0/351255536 >> v1:172.21.15.192:6789/0 conn(0x7f70ac0d0f10 legacy=0x7f70ac15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.580+0000 7f70b1791640 1 -- 172.21.15.111:0/351255536 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac15e7c0 con 0x7f70ac146b10 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.581+0000 7f70b1791640 1 --2- 172.21.15.111:0/351255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac146b10 0x7f70ac161a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f709c009e50 tx=0x7f709c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900f0767d524bb60 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.581+0000 7f70abfff640 1 -- 172.21.15.111:0/351255536 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c02ca70 con 0x7f70ac146b10 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.581+0000 7f70abfff640 1 -- 172.21.15.111:0/351255536 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c02cbd0 con 0x7f70ac146b10 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.581+0000 7f70abfff640 1 -- 172.21.15.111:0/351255536 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0343f0 con 0x7f70ac146b10 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 -- 172.21.15.111:0/351255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac146b10 msgr2=0x7f70ac161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 --2- 172.21.15.111:0/351255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac146b10 0x7f70ac161a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f709c009e50 tx=0x7f709c02bed0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 -- 172.21.15.111:0/351255536 shutdown_connections 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 --2- 172.21.15.111:0/351255536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 --2- 172.21.15.111:0/351255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac146b10 0x7f70ac161a30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 -- 172.21.15.111:0/351255536 >> 172.21.15.111:0/351255536 conn(0x7f70ac070af0 msgr2=0x7f70ac0662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 -- 172.21.15.111:0/351255536 shutdown_connections 2023-12-04T05:39:41.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.582+0000 7f70b39f5640 1 -- 172.21.15.111:0/351255536 wait complete. 2023-12-04T05:39:41.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.583+0000 7f70b39f5640 1 Processor -- start 2023-12-04T05:39:41.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.583+0000 7f70b39f5640 1 -- start start 2023-12-04T05:39:41.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 0x7f70ac0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70ac146b10 0x7f70ac1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0cb400 con 0x7f70ac0d0f10 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac0cbd60 con 0x7f70ac146b10 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1f76b0 con 0x7f70ac15dfb0 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 0x7f70ac0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 0x7f70ac0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36552/0 (socket says 172.21.15.111:36552) 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 -- 172.21.15.111:0/4071663084 learned_addr learned my addr 172.21.15.111:0/4071663084 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1f92640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b0f90640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70ac146b10 0x7f70ac1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 -- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 msgr2=0x7f70ac144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 -- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70ac146b10 msgr2=0x7f70ac1f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70ac146b10 0x7f70ac1f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 -- 172.21.15.111:0/4071663084 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009b30 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b1791640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 0x7f70ac0cb820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f709c000f80 tx=0x7f709c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c009510 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c03e450 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac0ca780 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.584+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c03e800 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.585+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac0cad40 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.585+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac0b7350 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.586+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f709c03e960 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.586+0000 7f70a9ffb640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f709806b2a0 0x7f709806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.586+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f709c0ac9b0 con 0x7f70ac0d0f10 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.586+0000 7f70b0f90640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f709806b2a0 0x7f709806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.587+0000 7f70b0f90640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f709806b2a0 0x7f709806d760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0b6700 tx=0x7f70a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.591+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f709c077820 con 0x7f70ac0d0f10 2023-12-04T05:39:41.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.758+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70ac069360 con 0x7f70ac0d0f10 2023-12-04T05:39:41.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.758+0000 7f70a9ffb640 1 -- 172.21.15.111:0/4071663084 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f709c07ae40 con 0x7f70ac0d0f10 2023-12-04T05:39:41.759 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f709806b2a0 msgr2=0x7f709806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f709806b2a0 0x7f709806d760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0b6700 tx=0x7f70a0008040 comp rx=0 tx=0).stop 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 msgr2=0x7f70ac0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 0x7f70ac0cb820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f709c000f80 tx=0x7f709c004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 shutdown_connections 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f709806b2a0 0x7f709806d760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70ac15dfb0 0x7f70ac144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70ac146b10 0x7f70ac1f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 --2- 172.21.15.111:0/4071663084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70ac0d0f10 0x7f70ac0cb820 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 >> 172.21.15.111:0/4071663084 conn(0x7f70ac070af0 msgr2=0x7f70ac0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:41.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 shutdown_connections 2023-12-04T05:39:41.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.760+0000 7f70b39f5640 1 -- 172.21.15.111:0/4071663084 wait complete. 2023-12-04T05:39:41.772 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:41.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.873+0000 7f611fdc5640 1 Processor -- start 2023-12-04T05:39:41.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.874+0000 7f611fdc5640 1 -- start start 2023-12-04T05:39:41.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.874+0000 7f611fdc5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6118070110 0x7f611806f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.874+0000 7f611fdc5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f61180cc3a0 0x7f61180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.874+0000 7f611fdc5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61180cfd60 con 0x7f6118099a00 2023-12-04T05:39:41.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.874+0000 7f611fdc5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611809a5d0 con 0x7f6118070110 2023-12-04T05:39:41.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.874+0000 7f611fdc5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611809a8b0 con 0x7f61180cc3a0 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6118070110 0x7f611806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6118070110 0x7f611806f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36560/0 (socket says 172.21.15.111:36560) 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 -- 172.21.15.111:0/3037546034 learned_addr learned my addr 172.21.15.111:0/3037546034 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 -- 172.21.15.111:0/3037546034 >> v2:172.21.15.192:3300/0 conn(0x7f61180cc3a0 msgr2=0x7f61180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611e362640 1 --2- 172.21.15.111:0/3037546034 >> v2:172.21.15.192:3300/0 conn(0x7f61180cc3a0 0x7f61180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 --2- 172.21.15.111:0/3037546034 >> v2:172.21.15.192:3300/0 conn(0x7f61180cc3a0 0x7f61180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 -- 172.21.15.111:0/3037546034 >> v1:172.21.15.192:6789/0 conn(0x7f6118099a00 legacy=0x7f6118099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611d360640 1 -- 172.21.15.111:0/3037546034 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180cf560 con 0x7f6118070110 2023-12-04T05:39:41.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.875+0000 7f611e362640 1 --2- 172.21.15.111:0/3037546034 >> v2:172.21.15.192:3300/0 conn(0x7f61180cc3a0 0x7f61180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:41.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611d360640 1 --2- 172.21.15.111:0/3037546034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118070110 0x7f611806f600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6108009870 tx=0x7f610802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd61289c1611ad8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611cb5f640 1 -- 172.21.15.111:0/3037546034 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108031870 con 0x7f6118070110 2023-12-04T05:39:41.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611cb5f640 1 -- 172.21.15.111:0/3037546034 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61080319d0 con 0x7f6118070110 2023-12-04T05:39:41.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611cb5f640 1 -- 172.21.15.111:0/3037546034 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108031d60 con 0x7f6118070110 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 -- 172.21.15.111:0/3037546034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118070110 msgr2=0x7f611806f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 --2- 172.21.15.111:0/3037546034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118070110 0x7f611806f600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6108009870 tx=0x7f610802b910 comp rx=0 tx=0).stop 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 -- 172.21.15.111:0/3037546034 shutdown_connections 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 --2- 172.21.15.111:0/3037546034 >> v2:172.21.15.192:3300/0 conn(0x7f61180cc3a0 0x7f61180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 --2- 172.21.15.111:0/3037546034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118070110 0x7f611806f600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 -- 172.21.15.111:0/3037546034 >> 172.21.15.111:0/3037546034 conn(0x7f61180b2a70 msgr2=0x7f61180b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 -- 172.21.15.111:0/3037546034 shutdown_connections 2023-12-04T05:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.876+0000 7f611fdc5640 1 -- 172.21.15.111:0/3037546034 wait complete. 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.877+0000 7f611fdc5640 1 Processor -- start 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 -- start start 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6118070110 0x7f61180b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 0x7f61180b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61180cc3a0 0x7f61180b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180c5960 con 0x7f6118099a00 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180c5c40 con 0x7f6118070110 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611fdc5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61180c5f20 con 0x7f61180cc3a0 2023-12-04T05:39:41.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611d360640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 0x7f61180b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611d360640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 0x7f61180b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36568/0 (socket says 172.21.15.111:36568) 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.878+0000 7f611d360640 1 -- 172.21.15.111:0/315169288 learned_addr learned my addr 172.21.15.111:0/315169288 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611e362640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61180cc3a0 0x7f61180b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611d360640 1 -- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61180cc3a0 msgr2=0x7f61180b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611db61640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6118070110 0x7f61180b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611d360640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61180cc3a0 0x7f61180b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611d360640 1 -- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6118070110 msgr2=0x7f61180b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611d360640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6118070110 0x7f61180b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611d360640 1 -- 172.21.15.111:0/315169288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080094d0 con 0x7f6118099a00 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611e362640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61180cc3a0 0x7f61180b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611db61640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6118070110 0x7f61180b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611d360640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 0x7f61180b3e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6108009870 tx=0x7f61080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108031820 con 0x7f6118099a00 2023-12-04T05:39:41.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61080443f0 con 0x7f6118099a00 2023-12-04T05:39:41.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61180c6200 con 0x7f6118099a00 2023-12-04T05:39:41.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61080447a0 con 0x7f6118099a00 2023-12-04T05:39:41.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.879+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61180b5fd0 con 0x7f6118099a00 2023-12-04T05:39:41.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.880+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61180c8d40 con 0x7f6118099a00 2023-12-04T05:39:41.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.880+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f610804d450 con 0x7f6118099a00 2023-12-04T05:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.881+0000 7f610e7fc640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f60fc06b2a0 0x7f60fc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.881+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f610802d040 con 0x7f6118099a00 2023-12-04T05:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.881+0000 7f611db61640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f60fc06b2a0 0x7f60fc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.881+0000 7f611db61640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f60fc06b2a0 0x7f60fc06d760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6114003eb0 tx=0x7f6114009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:41.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:41.885+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61180c8d40 con 0x7f6118099a00 2023-12-04T05:39:42.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.051+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61180c8d40 con 0x7f6118099a00 2023-12-04T05:39:42.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.052+0000 7f610e7fc640 1 -- 172.21.15.111:0/315169288 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61180c8d40 con 0x7f6118099a00 2023-12-04T05:39:42.052 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:42.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f60fc06b2a0 msgr2=0x7f60fc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f60fc06b2a0 0x7f60fc06d760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6114003eb0 tx=0x7f6114009040 comp rx=0 tx=0).stop 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 msgr2=0x7f61180b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 0x7f61180b3e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6108009870 tx=0x7f61080046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 shutdown_connections 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f60fc06b2a0 0x7f60fc06d760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61180cc3a0 0x7f61180b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6118099a00 0x7f61180b3e40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 --2- 172.21.15.111:0/315169288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6118070110 0x7f61180b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 >> 172.21.15.111:0/315169288 conn(0x7f61180b2a70 msgr2=0x7f611815cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 shutdown_connections 2023-12-04T05:39:42.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.054+0000 7f611fdc5640 1 -- 172.21.15.111:0/315169288 wait complete. 2023-12-04T05:39:42.071 INFO:tasks.cephfs_test_runner:Starting test: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a client unmounts, the thread count on the MDS goes back 2023-12-04T05:39:42.071 INFO:tasks.cephfs_test_runner:test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:39:42.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:42.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.171+0000 7f26f650f640 1 Processor -- start 2023-12-04T05:39:42.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.171+0000 7f26f650f640 1 -- start start 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f650f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26f006cae0 0x7f26f006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f650f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 0x7f26f0159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f650f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f0154220 con 0x7f26f015dab0 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f650f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f015d2f0 con 0x7f26f006cae0 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f650f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f006d400 con 0x7f26f00b3310 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f4aac640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f26f015dab0 0x7f26f00703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37854/0 (socket says 172.21.15.111:37854) 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26f4aac640 1 -- 172.21.15.111:0/2295274593 learned_addr learned my addr 172.21.15.111:0/2295274593 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26ef7fe640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 0x7f26f0159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084202983 0 0) 0x7f26e8001490 con 0x7f26f015dab0 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26effff640 1 --2- 172.21.15.111:0/2295274593 >> v2:172.21.15.192:3300/0 conn(0x7f26f006cae0 0x7f26f006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d8004630 con 0x7f26f015dab0 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 903750120 0 0) 0x7f26d8004630 con 0x7f26f015dab0 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 msgr2=0x7f26f0159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 0x7f26f0159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 >> v2:172.21.15.192:3300/0 conn(0x7f26f006cae0 msgr2=0x7f26f006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 --2- 172.21.15.111:0/2295274593 >> v2:172.21.15.192:3300/0 conn(0x7f26f006cae0 0x7f26f006cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f015a5c0 con 0x7f26f015dab0 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f26e8001b80 con 0x7f26f015dab0 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26effff640 1 --2- 172.21.15.111:0/2295274593 >> v2:172.21.15.192:3300/0 conn(0x7f26f006cae0 0x7f26f006cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 >> v1:172.21.15.111:6789/0 conn(0x7f26f015dab0 legacy=0x7f26f00703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8004a80 0x7f26d8004ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 0x7f26d80058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d8005e10 0x7f26d8006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f015d2f0 con 0x7f26d8004a80 2023-12-04T05:39:42.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f006d400 con 0x7f26d8005e10 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.172+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d800ca90 con 0x7f26f00b3310 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26effff640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8004a80 0x7f26d8004ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26ef7fe640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 0x7f26d80058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26f4aac640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d8005e10 0x7f26d8006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26effff640 1 -- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 msgr2=0x7f26d80058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26effff640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f00b3310 0x7f26d80058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26effff640 1 -- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d8005e10 msgr2=0x7f26d8006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26effff640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d8005e10 0x7f26d8006290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26effff640 1 -- 172.21.15.111:0/2295274593 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8004630 con 0x7f26d8004a80 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.173+0000 7f26f4aac640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d8005e10 0x7f26d8006290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26effff640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8004a80 0x7f26d8004ea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26e40098d0 tx=0x7f26d8006900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24e95736134640a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4004230 con 0x7f26d8004a80 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4004390 con 0x7f26d8004a80 2023-12-04T05:39:42.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26eeffd640 1 -- 172.21.15.111:0/2295274593 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4003830 con 0x7f26d8004a80 2023-12-04T05:39:42.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f650f640 1 -- 172.21.15.111:0/2295274593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8004a80 msgr2=0x7f26d8004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f650f640 1 --2- 172.21.15.111:0/2295274593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d8004a80 0x7f26d8004ea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26e40098d0 tx=0x7f26d8006900 comp rx=0 tx=0).stop 2023-12-04T05:39:42.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f4aac640 1 -- 172.21.15.111:0/2295274593 reap_dead start 2023-12-04T05:39:42.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f650f640 1 -- 172.21.15.111:0/2295274593 shutdown_connections 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f650f640 1 -- 172.21.15.111:0/2295274593 >> 172.21.15.111:0/2295274593 conn(0x7f26f006a4c0 msgr2=0x7f26f006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f650f640 1 -- 172.21.15.111:0/2295274593 shutdown_connections 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.174+0000 7f26f650f640 1 -- 172.21.15.111:0/2295274593 wait complete. 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.175+0000 7f26f650f640 1 Processor -- start 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 -- start start 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f0145820 0x7f26f01460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f01465f0 0x7f26f0155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 0x7f26f00adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00ada70 con 0x7f26f0155660 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00acb80 con 0x7f26f01465f0 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f650f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00ace60 con 0x7f26f0145820 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f4aac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 0x7f26f00adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f4aac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 0x7f26f00adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36576/0 (socket says 172.21.15.111:36576) 2023-12-04T05:39:42.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.176+0000 7f26f4aac640 1 -- 172.21.15.111:0/2224275359 learned_addr learned my addr 172.21.15.111:0/2224275359 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f4aac640 1 -- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f0145820 msgr2=0x7f26f01460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26effff640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f0145820 0x7f26f01460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f4aac640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f0145820 0x7f26f01460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26ef7fe640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f01465f0 0x7f26f0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f4aac640 1 -- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f01465f0 msgr2=0x7f26f0155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f4aac640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f01465f0 0x7f26f0155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f4aac640 1 -- 172.21.15.111:0/2224275359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00ad140 con 0x7f26f0155660 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26effff640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f0145820 0x7f26f01460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f4aac640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 0x7f26f00adf00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001330 tx=0x7f26e800e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800eaf0 con 0x7f26f0155660 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800ec70 con 0x7f26f0155660 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f00ad420 con 0x7f26f0155660 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8013420 con 0x7f26f0155660 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.177+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f0145c00 con 0x7f26f0155660 2023-12-04T05:39:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.178+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f00afde0 con 0x7f26f0155660 2023-12-04T05:39:42.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.178+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f26e8013580 con 0x7f26f0155660 2023-12-04T05:39:42.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.179+0000 7f26ecff9640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26dc06b2a0 0x7f26dc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.179+0000 7f26effff640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26dc06b2a0 0x7f26dc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.179+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26e8089ec0 con 0x7f26f0155660 2023-12-04T05:39:42.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.179+0000 7f26effff640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26dc06b2a0 0x7f26dc06d760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26e4035000 tx=0x7f26e4039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.183+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26e8054d30 con 0x7f26f0155660 2023-12-04T05:39:42.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.353+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f0154220 con 0x7f26f0155660 2023-12-04T05:39:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.354+0000 7f26ecff9640 1 -- 172.21.15.111:0/2224275359 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f26f0154220 con 0x7f26f0155660 2023-12-04T05:39:42.355 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:42.356 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26dc06b2a0 msgr2=0x7f26dc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26dc06b2a0 0x7f26dc06d760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26e4035000 tx=0x7f26e4039040 comp rx=0 tx=0).stop 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 msgr2=0x7f26f00adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 0x7f26f00adf00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001330 tx=0x7f26e800e160 comp rx=0 tx=0).stop 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 shutdown_connections 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26dc06b2a0 0x7f26dc06d760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f0155660 0x7f26f00adf00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26f01465f0 0x7f26f0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 --2- 172.21.15.111:0/2224275359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26f0145820 0x7f26f01460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 >> 172.21.15.111:0/2224275359 conn(0x7f26f006a4c0 msgr2=0x7f26f0153920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 shutdown_connections 2023-12-04T05:39:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.356+0000 7f26f650f640 1 -- 172.21.15.111:0/2224275359 wait complete. 2023-12-04T05:39:42.368 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:42.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:42.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.470+0000 7f3bc5d05640 1 Processor -- start 2023-12-04T05:39:42.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.470+0000 7f3bc5d05640 1 -- start start 2023-12-04T05:39:42.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bc5d05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3bc00d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bc5d05640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3bc00d01e0 0x7f3bc0160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bc5d05640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc015ddc0 con 0x7f3bc00d1310 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bc5d05640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc015e0a0 con 0x7f3bc00d01e0 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bc5d05640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc015e380 con 0x7f3bc015d480 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bbeffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3bc00d1310 0x7f3bc00cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37856/0 (socket says 172.21.15.111:37856) 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.471+0000 7f3bbeffd640 1 -- 172.21.15.111:0/489076886 learned_addr learned my addr 172.21.15.111:0/489076886 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 --2- 172.21.15.111:0/489076886 >> v2:172.21.15.192:3300/0 conn(0x7f3bc00d01e0 0x7f3bc0160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbf7fe640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3bc00d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089955851 0 0) 0x7f3bc015ddc0 con 0x7f3bc00d1310 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba8003520 con 0x7f3bc00d1310 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028578516 0 0) 0x7f3ba8003520 con 0x7f3bc00d1310 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 msgr2=0x7f3bc00d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3bc00d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 >> v2:172.21.15.192:3300/0 conn(0x7f3bc00d01e0 msgr2=0x7f3bc0160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 --2- 172.21.15.111:0/489076886 >> v2:172.21.15.192:3300/0 conn(0x7f3bc00d01e0 0x7f3bc0160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc015e660 con 0x7f3bc00d1310 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f3bb4001670 con 0x7f3bc00d1310 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 >> v1:172.21.15.111:6789/0 conn(0x7f3bc00d1310 legacy=0x7f3bc00cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3ba8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ba8003ef0 0x7f3ba80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba8004d00 0x7f3ba8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc015e0a0 con 0x7f3ba8004d00 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc015e380 con 0x7f3ba8003ef0 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba800b980 con 0x7f3bc015d480 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba8004d00 0x7f3ba8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbf7fe640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3ba8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbeffd640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ba8003ef0 0x7f3ba80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 -- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 msgr2=0x7f3ba8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3ba8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 -- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ba8003ef0 msgr2=0x7f3ba80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ba8003ef0 0x7f3ba80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbffff640 1 -- 172.21.15.111:0/489076886 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8003520 con 0x7f3ba8004d00 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbeffd640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ba8003ef0 0x7f3ba80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.472+0000 7f3bbf7fe640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc015d480 0x7f3ba8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.473+0000 7f3bbffff640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba8004d00 0x7f3ba8005180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3bc008c930 tx=0x7f3ba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2de6409217cddfa server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.473+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb802cb90 con 0x7f3ba8004d00 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.473+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb800f040 con 0x7f3ba8004d00 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.473+0000 7f3bbe7fc640 1 -- 172.21.15.111:0/489076886 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8034440 con 0x7f3ba8004d00 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bc5d05640 1 -- 172.21.15.111:0/489076886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba8004d00 msgr2=0x7f3ba8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/489076886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ba8004d00 0x7f3ba8005180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3bc008c930 tx=0x7f3ba80057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bbffff640 1 -- 172.21.15.111:0/489076886 reap_dead start 2023-12-04T05:39:42.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bc5d05640 1 -- 172.21.15.111:0/489076886 shutdown_connections 2023-12-04T05:39:42.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bc5d05640 1 -- 172.21.15.111:0/489076886 >> 172.21.15.111:0/489076886 conn(0x7f3bc009c540 msgr2=0x7f3bc00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bc5d05640 1 -- 172.21.15.111:0/489076886 shutdown_connections 2023-12-04T05:39:42.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.474+0000 7f3bc5d05640 1 -- 172.21.15.111:0/489076886 wait complete. 2023-12-04T05:39:42.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.475+0000 7f3bc5d05640 1 Processor -- start 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.475+0000 7f3bc5d05640 1 -- start start 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bc5d05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc0151cd0 0x7f3bc0158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bc5d05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc006c2b0 0x7f3bc006c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bc5d05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 0x7f3bc0156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bc5d05640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01588c0 con 0x7f3bc0156650 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bc5d05640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc006cc70 con 0x7f3bc0151cd0 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bc5d05640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc00c6fe0 con 0x7f3bc006c2b0 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 0x7f3bc0156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 0x7f3bc0156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36584/0 (socket says 172.21.15.111:36584) 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 -- 172.21.15.111:0/2724229507 learned_addr learned my addr 172.21.15.111:0/2724229507 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbeffd640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc006c2b0 0x7f3bc006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbf7fe640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc0151cd0 0x7f3bc0158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 -- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc006c2b0 msgr2=0x7f3bc006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc006c2b0 0x7f3bc006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 -- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc0151cd0 msgr2=0x7f3bc0158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc0151cd0 0x7f3bc0158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 -- 172.21.15.111:0/2724229507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc00b7ff0 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.476+0000 7f3bbffff640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 0x7f3bc0156ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8037040 tx=0x7f3bb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.477+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8034cd0 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.477+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8034e30 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.477+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc00b82d0 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.477+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb803e660 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.477+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc00b7c10 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.478+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc0067980 con 0x7f3bc0156650 2023-12-04T05:39:42.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.478+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3bb803e7c0 con 0x7f3bc0156650 2023-12-04T05:39:42.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.478+0000 7f3ba7fff640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3bac06b2d0 0x7f3bac06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.479+0000 7f3bbf7fe640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3bac06b2d0 0x7f3bac06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.479+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bb80aba10 con 0x7f3bc0156650 2023-12-04T05:39:42.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.479+0000 7f3bbf7fe640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3bac06b2d0 0x7f3bac06d790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00b8e70 tx=0x7f3bb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.483+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bb80768b0 con 0x7f3bc0156650 2023-12-04T05:39:42.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.649+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bc00c7150 con 0x7f3bc0156650 2023-12-04T05:39:42.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.650+0000 7f3ba7fff640 1 -- 172.21.15.111:0/2724229507 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bb80390a0 con 0x7f3bc0156650 2023-12-04T05:39:42.651 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:42.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3bac06b2d0 msgr2=0x7f3bac06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3bac06b2d0 0x7f3bac06d790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00b8e70 tx=0x7f3bb0008040 comp rx=0 tx=0).stop 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 msgr2=0x7f3bc0156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 0x7f3bc0156ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8037040 tx=0x7f3bb8009750 comp rx=0 tx=0).stop 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 shutdown_connections 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3bac06b2d0 0x7f3bac06d790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bc0156650 0x7f3bc0156ad0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc006c2b0 0x7f3bc006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 --2- 172.21.15.111:0/2724229507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc0151cd0 0x7f3bc0158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 >> 172.21.15.111:0/2724229507 conn(0x7f3bc009c540 msgr2=0x7f3bc0160740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 shutdown_connections 2023-12-04T05:39:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.652+0000 7f3bc5d05640 1 -- 172.21.15.111:0/2724229507 wait complete. 2023-12-04T05:39:42.664 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 Processor -- start 2023-12-04T05:39:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 -- start start 2023-12-04T05:39:42.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f54d813f640 0x7f54d813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 0x7f54d8072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d813dd10 con 0x7f54d813cee0 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d8072670 con 0x7f54d813f640 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.765+0000 7f54ddd0f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d813d2c0 con 0x7f54d8071d20 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f54d813cee0 0x7f54d813d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37862/0 (socket says 172.21.15.111:37862) 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d77fe640 1 -- 172.21.15.111:0/1112053545 learned_addr learned my addr 172.21.15.111:0/1112053545 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802905478 0 0) 0x7f54d813dd10 con 0x7f54d813cee0 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d7fff640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 0x7f54d8072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c0005730 con 0x7f54d813cee0 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1112053545 >> v2:172.21.15.192:3300/0 conn(0x7f54d813f640 0x7f54d813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488941854 0 0) 0x7f54c0005730 con 0x7f54d813cee0 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 msgr2=0x7f54d8072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 0x7f54d8072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 >> v2:172.21.15.192:3300/0 conn(0x7f54d813f640 msgr2=0x7f54d813fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 --2- 172.21.15.111:0/1112053545 >> v2:172.21.15.192:3300/0 conn(0x7f54d813f640 0x7f54d813fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d81400e0 con 0x7f54d813cee0 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f54c8001670 con 0x7f54d813cee0 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 >> v1:172.21.15.111:6789/0 conn(0x7f54d813cee0 legacy=0x7f54d813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 0x7f54c0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54c00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54c0006f10 0x7f54c0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8072670 con 0x7f54c0006100 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d813d2c0 con 0x7f54c0006f10 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c000dc70 con 0x7f54d8071d20 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54c00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d77fe640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 0x7f54c0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.766+0000 7f54d7fff640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54c0006f10 0x7f54c0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d6ffd640 1 -- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 msgr2=0x7f54c0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d8071d20 0x7f54c0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d6ffd640 1 -- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54c0006f10 msgr2=0x7f54c0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54c0006f10 0x7f54c0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d6ffd640 1 -- 172.21.15.111:0/1112053545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c0005730 con 0x7f54c0006100 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d7fff640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54c0006f10 0x7f54c0007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54c00069d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009ac0 tx=0x7f54c0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0d878a54482bb server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc02c710 con 0x7f54c0006100 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc02c870 con 0x7f54c0006100 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.767+0000 7f54d67fc640 1 -- 172.21.15.111:0/1112053545 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc02cc00 con 0x7f54c0006100 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1112053545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 msgr2=0x7f54c00069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1112053545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54c00069d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009ac0 tx=0x7f54c0007a00 comp rx=0 tx=0).stop 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54d7fff640 1 -- 172.21.15.111:0/1112053545 reap_dead start 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1112053545 shutdown_connections 2023-12-04T05:39:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1112053545 >> 172.21.15.111:0/1112053545 conn(0x7f54d80b0cf0 msgr2=0x7f54d80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1112053545 shutdown_connections 2023-12-04T05:39:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.768+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1112053545 wait complete. 2023-12-04T05:39:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.769+0000 7f54ddd0f640 1 Processor -- start 2023-12-04T05:39:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.769+0000 7f54ddd0f640 1 -- start start 2023-12-04T05:39:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54ddd0f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54d80b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54ddd0f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80b3410 0x7f54d8138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54ddd0f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d806c5c0 0x7f54d806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54ddd0f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80b4f70 con 0x7f54c0006100 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54ddd0f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d806cf90 con 0x7f54d806c5c0 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54ddd0f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8138940 con 0x7f54d80b3410 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54d80b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54d80b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36608/0 (socket says 172.21.15.111:36608) 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 -- 172.21.15.111:0/1934964202 learned_addr learned my addr 172.21.15.111:0/1934964202 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80b3410 0x7f54d8138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d7fff640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d806c5c0 0x7f54d806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 -- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80b3410 msgr2=0x7f54d8138400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80b3410 0x7f54d8138400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 -- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d806c5c0 msgr2=0x7f54d806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d806c5c0 0x7f54d806ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d77fe640 1 -- 172.21.15.111:0/1934964202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d8138cd0 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.770+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80b3410 0x7f54d8138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.771+0000 7f54d77fe640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54d80b3c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54c8001cf0 tx=0x7f54c800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.771+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c800ee60 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.771+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c800c840 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.771+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d80b42f0 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.771+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c800cbf0 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.771+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d80b48b0 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.772+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d8069360 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.772+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f54c800cd50 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.773+0000 7f54bffff640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f54c406b2a0 0x7f54c406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.773+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54c8089b40 con 0x7f54c0006100 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.773+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f54c406b2a0 0x7f54c406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:42.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.773+0000 7f54d6ffd640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f54c406b2a0 0x7f54c406d760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54cc036000 tx=0x7f54cc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:42.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.777+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54c8054930 con 0x7f54c0006100 2023-12-04T05:39:42.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.945+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54d8070380 con 0x7f54c0006100 2023-12-04T05:39:42.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.945+0000 7f54bffff640 1 -- 172.21.15.111:0/1934964202 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54c8057f50 con 0x7f54c0006100 2023-12-04T05:39:42.947 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f54c406b2a0 msgr2=0x7f54c406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f54c406b2a0 0x7f54c406d760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54cc036000 tx=0x7f54cc030000 comp rx=0 tx=0).stop 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 msgr2=0x7f54d80b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54d80b3c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54c8001cf0 tx=0x7f54c800ec10 comp rx=0 tx=0).stop 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 shutdown_connections 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f54c406b2a0 0x7f54c406d760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d806c5c0 0x7f54d806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80b3410 0x7f54d8138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 --2- 172.21.15.111:0/1934964202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c0006100 0x7f54d80b3c60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 >> 172.21.15.111:0/1934964202 conn(0x7f54d80b0cf0 msgr2=0x7f54d80aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 shutdown_connections 2023-12-04T05:39:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:42.947+0000 7f54ddd0f640 1 -- 172.21.15.111:0/1934964202 wait complete. 2023-12-04T05:39:42.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:43.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.060+0000 7f3f3fd33640 1 Processor -- start 2023-12-04T05:39:43.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.061+0000 7f3f3fd33640 1 -- start start 2023-12-04T05:39:43.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.061+0000 7f3f3fd33640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3f380af750 0x7f3f380ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.061+0000 7f3f3fd33640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f3815da80 0x7f3f380afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.061+0000 7f3f3fd33640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3815df30 con 0x7f3f380ac100 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.061+0000 7f3f3fd33640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f38157f60 con 0x7f3f3815da80 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.061+0000 7f3f3fd33640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f380cd190 con 0x7f3f380af750 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f3815da80 0x7f3f380afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f3815da80 0x7f3f380afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36622/0 (socket says 172.21.15.111:36622) 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/4093825782 learned_addr learned my addr 172.21.15.111:0/4093825782 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3dacf640 1 --2- 172.21.15.111:0/4093825782 >> v2:172.21.15.192:3300/0 conn(0x7f3f380af750 0x7f3f380ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/4093825782 >> v2:172.21.15.192:3300/0 conn(0x7f3f380af750 msgr2=0x7f3f380ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 --2- 172.21.15.111:0/4093825782 >> v2:172.21.15.192:3300/0 conn(0x7f3f380af750 0x7f3f380ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/4093825782 >> v1:172.21.15.192:6789/0 conn(0x7f3f380ac100 legacy=0x7f3f380cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.062+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/4093825782 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380b0070 con 0x7f3f3815da80 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3d2ce640 1 --2- 172.21.15.111:0/4093825782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3815da80 0x7f3f380afb30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f3806bdc0 tx=0x7f3f2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d2fb22ed80e3da server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3cacd640 1 -- 172.21.15.111:0/4093825782 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28031870 con 0x7f3f3815da80 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3cacd640 1 -- 172.21.15.111:0/4093825782 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f280319d0 con 0x7f3f3815da80 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3cacd640 1 -- 172.21.15.111:0/4093825782 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28031d60 con 0x7f3f3815da80 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 -- 172.21.15.111:0/4093825782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3815da80 msgr2=0x7f3f380afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/4093825782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3815da80 0x7f3f380afb30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f3806bdc0 tx=0x7f3f2802b910 comp rx=0 tx=0).stop 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 -- 172.21.15.111:0/4093825782 shutdown_connections 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/4093825782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3815da80 0x7f3f380afb30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/4093825782 >> v2:172.21.15.192:3300/0 conn(0x7f3f380af750 0x7f3f380ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 -- 172.21.15.111:0/4093825782 >> 172.21.15.111:0/4093825782 conn(0x7f3f380b4640 msgr2=0x7f3f380b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:43.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 -- 172.21.15.111:0/4093825782 shutdown_connections 2023-12-04T05:39:43.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.063+0000 7f3f3fd33640 1 -- 172.21.15.111:0/4093825782 wait complete. 2023-12-04T05:39:43.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.064+0000 7f3f3fd33640 1 Processor -- start 2023-12-04T05:39:43.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.064+0000 7f3f3fd33640 1 -- start start 2023-12-04T05:39:43.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3fd33640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f380ac100 0x7f3f381466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3fd33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 0x7f3f380b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3fd33640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f3815da80 0x7f3f380b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3fd33640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380b4450 con 0x7f3f380af750 2023-12-04T05:39:43.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3fd33640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380b2610 con 0x7f3f3815da80 2023-12-04T05:39:43.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3fd33640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380b28f0 con 0x7f3f380ac100 2023-12-04T05:39:43.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3d2ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 0x7f3f380b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3d2ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 0x7f3f380b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36628/0 (socket says 172.21.15.111:36628) 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.065+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/3761592259 learned_addr learned my addr 172.21.15.111:0/3761592259 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3dacf640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f380ac100 0x7f3f381466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f380ac100 msgr2=0x7f3f381466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3e2d0640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f3815da80 0x7f3f380b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3d2ce640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f380ac100 0x7f3f381466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f3815da80 msgr2=0x7f3f380b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3d2ce640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f3815da80 0x7f3f380b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3d2ce640 1 -- 172.21.15.111:0/3761592259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f280094d0 con 0x7f3f380af750 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3dacf640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f380ac100 0x7f3f381466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3e2d0640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f3815da80 0x7f3f380b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3d2ce640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 0x7f3f380b1f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3f2802b950 tx=0x7f3f280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2803ad30 con 0x7f3f380af750 2023-12-04T05:39:43.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f280443f0 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f380b2bd0 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280447a0 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.066+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f380b3190 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.067+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f380b3380 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.067+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3f2804d450 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.068+0000 7f3f2e7fc640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f1c06b2a0 0x7f3f1c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.068+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f2802d040 con 0x7f3f380af750 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.068+0000 7f3f3dacf640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f1c06b2a0 0x7f3f1c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.068+0000 7f3f3dacf640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f1c06b2a0 0x7f3f1c06d760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006fd0 tx=0x7f3f34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:43.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.072+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f28076f80 con 0x7f3f380af750 2023-12-04T05:39:43.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.237+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f38069360 con 0x7f3f380af750 2023-12-04T05:39:43.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.237+0000 7f3f2e7fc640 1 -- 172.21.15.111:0/3761592259 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f28035020 con 0x7f3f380af750 2023-12-04T05:39:43.239 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f1c06b2a0 msgr2=0x7f3f1c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f1c06b2a0 0x7f3f1c06d760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006fd0 tx=0x7f3f34008040 comp rx=0 tx=0).stop 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 msgr2=0x7f3f380b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 0x7f3f380b1f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3f2802b950 tx=0x7f3f280046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 shutdown_connections 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f1c06b2a0 0x7f3f1c06d760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f3815da80 0x7f3f380b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f380af750 0x7f3f380b1f10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 --2- 172.21.15.111:0/3761592259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f380ac100 0x7f3f381466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 >> 172.21.15.111:0/3761592259 conn(0x7f3f380b4640 msgr2=0x7f3f380ac4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:43.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 shutdown_connections 2023-12-04T05:39:43.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.239+0000 7f3f3fd33640 1 -- 172.21.15.111:0/3761592259 wait complete. 2023-12-04T05:39:43.252 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2023-12-04T05:39:43.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.351+0000 7f6a421a9640 1 Processor -- start 2023-12-04T05:39:43.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.351+0000 7f6a421a9640 1 -- start start 2023-12-04T05:39:43.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.351+0000 7f6a421a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 0x7f6a3c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.351+0000 7f6a421a9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c154220 con 0x7f6a3c15dab0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.351+0000 7f6a421a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c15d2f0 con 0x7f6a3c0b3310 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.351+0000 7f6a421a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c15a3c0 con 0x7f6a3c06cae0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6a3c15dab0 0x7f6a3c0703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37866/0 (socket says 172.21.15.111:37866) 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3bfff640 1 -- 172.21.15.111:0/429648827 learned_addr learned my addr 172.21.15.111:0/429648827 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3b7fe640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 0x7f6a3c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631114456 0 0) 0x7f6a34001490 con 0x7f6a3c15dab0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a24003520 con 0x7f6a3c15dab0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799013603 0 0) 0x7f6a24003520 con 0x7f6a3c15dab0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 msgr2=0x7f6a3c06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 0x7f6a3c06cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 >> v1:172.21.15.192:6789/0 conn(0x7f6a3c0b3310 legacy=0x7f6a3c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c06d400 con 0x7f6a3c15dab0 2023-12-04T05:39:43.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f6a34001b80 con 0x7f6a3c15dab0 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 >> v1:172.21.15.111:6789/0 conn(0x7f6a3c15dab0 legacy=0x7f6a3c0703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a240039a0 0x7f6a24003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 0x7f6a240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c15a3c0 con 0x7f6a24004d00 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2400b980 con 0x7f6a240039a0 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2400bb40 con 0x7f6a3c06cae0 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3bfff640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3affd640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 0x7f6a240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.352+0000 7f6a3b7fe640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a240039a0 0x7f6a24003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3bfff640 1 -- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 msgr2=0x7f6a240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3bfff640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c06cae0 0x7f6a240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3bfff640 1 -- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a240039a0 msgr2=0x7f6a24003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3bfff640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a240039a0 0x7f6a24003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3bfff640 1 -- 172.21.15.111:0/429648827 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24003520 con 0x7f6a24004d00 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3bfff640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400a830 tx=0x7f6a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde6a5c6eb849719 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.353+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34034630 con 0x7f6a24004d00 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3402f070 con 0x7f6a24004d00 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a3a7fc640 1 -- 172.21.15.111:0/429648827 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a340349d0 con 0x7f6a24004d00 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a421a9640 1 -- 172.21.15.111:0/429648827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 msgr2=0x7f6a24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a421a9640 1 --2- 172.21.15.111:0/429648827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400a830 tx=0x7f6a240057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a3bfff640 1 -- 172.21.15.111:0/429648827 reap_dead start 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a421a9640 1 -- 172.21.15.111:0/429648827 shutdown_connections 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a421a9640 1 -- 172.21.15.111:0/429648827 >> 172.21.15.111:0/429648827 conn(0x7f6a3c06a4c0 msgr2=0x7f6a3c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a421a9640 1 -- 172.21.15.111:0/429648827 shutdown_connections 2023-12-04T05:39:43.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.354+0000 7f6a421a9640 1 -- 172.21.15.111:0/429648827 wait complete. 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.355+0000 7f6a421a9640 1 Processor -- start 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.355+0000 7f6a421a9640 1 -- start start 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a421a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a3c142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a421a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c158e90 0x7f6a3c159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a421a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a3c15adc0 0x7f6a3c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a421a9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c15b780 con 0x7f6a24004d00 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a421a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c148620 con 0x7f6a3c15adc0 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a421a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c148870 con 0x7f6a3c158e90 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a3c142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a3c142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36646/0 (socket says 172.21.15.111:36646) 2023-12-04T05:39:43.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 -- 172.21.15.111:0/905305230 learned_addr learned my addr 172.21.15.111:0/905305230 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 -- 172.21.15.111:0/905305230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c158e90 msgr2=0x7f6a3c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c158e90 0x7f6a3c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 -- 172.21.15.111:0/905305230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a3c15adc0 msgr2=0x7f6a3c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a3c15adc0 0x7f6a3c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.356+0000 7f6a3b7fe640 1 -- 172.21.15.111:0/905305230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c148b50 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.357+0000 7f6a3b7fe640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a3c142eb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0119a0 tx=0x7f6a2c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.357+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00fa10 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.357+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00fb70 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.357+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00fee0 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.357+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c148e30 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.357+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c149630 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.358+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c0693d0 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.358+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6a2c014440 con 0x7f6a24004d00 2023-12-04T05:39:43.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.358+0000 7f6a23fff640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6a2806b2a0 0x7f6a2806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:43.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.359+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a2c08a3a0 con 0x7f6a24004d00 2023-12-04T05:39:43.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.359+0000 7f6a3affd640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6a2806b2a0 0x7f6a2806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:43.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.359+0000 7f6a3affd640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6a2806b2a0 0x7f6a2806d760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c15ba70 tx=0x7f6a30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:43.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.363+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a2c055210 con 0x7f6a24004d00 2023-12-04T05:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:43.593+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) v1 -- 0x7f6a3c14de20 con 0x7f6a24004d00 2023-12-04T05:39:44.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.291+0000 7f6a23fff640 1 -- 172.21.15.111:0/905305230 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v46) v1 ==== 147+0+0 (secure 0 0 0) 0x7f6a2c058830 con 0x7f6a24004d00 2023-12-04T05:39:44.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6a2806b2a0 msgr2=0x7f6a2806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6a2806b2a0 0x7f6a2806d760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c15ba70 tx=0x7f6a30009040 comp rx=0 tx=0).stop 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 msgr2=0x7f6a3c142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a3c142eb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0119a0 tx=0x7f6a2c011e60 comp rx=0 tx=0).stop 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 shutdown_connections 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6a2806b2a0 0x7f6a2806d760 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a3c15adc0 0x7f6a3c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a3c158e90 0x7f6a3c159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 --2- 172.21.15.111:0/905305230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a24004d00 0x7f6a3c142eb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 >> 172.21.15.111:0/905305230 conn(0x7f6a3c06a4c0 msgr2=0x7f6a3c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 shutdown_connections 2023-12-04T05:39:44.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.295+0000 7f6a421a9640 1 -- 172.21.15.111:0/905305230 wait complete. 2023-12-04T05:39:44.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-04T05:39:44.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.415+0000 7f063abe5640 1 Processor -- start 2023-12-04T05:39:44.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.415+0000 7f063abe5640 1 -- start start 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f063abe5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0634071c30 0x7f06340cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f063abe5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06340724c0 con 0x7f06341438f0 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f063abe5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06340729f0 con 0x7f0634143390 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f063abe5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0634143770 con 0x7f0634071c30 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f0633fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f06341438f0 0x7f0634071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37876/0 (socket says 172.21.15.111:37876) 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f0633fff640 1 -- 172.21.15.111:0/3023739373 learned_addr learned my addr 172.21.15.111:0/3023739373 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f0639182640 1 --2- 172.21.15.111:0/3023739373 >> v2:172.21.15.192:3300/0 conn(0x7f0634071c30 0x7f06340cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793979345 0 0) 0x7f06340724c0 con 0x7f06341438f0 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0624003520 con 0x7f06341438f0 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26846459 0 0) 0x7f06340729f0 con 0x7f0634143390 2023-12-04T05:39:44.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06340724c0 con 0x7f0634143390 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942191569 0 0) 0x7f0624003520 con 0x7f06341438f0 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 >> v2:172.21.15.192:3300/0 conn(0x7f0634071c30 msgr2=0x7f06340cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 --2- 172.21.15.111:0/3023739373 >> v2:172.21.15.192:3300/0 conn(0x7f0634071c30 0x7f06340cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 >> v1:172.21.15.192:6789/0 conn(0x7f0634143390 legacy=0x7f06340cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063406e540 con 0x7f06341438f0 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f0620001670 con 0x7f06341438f0 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f0639182640 1 --2- 172.21.15.111:0/3023739373 >> v2:172.21.15.192:3300/0 conn(0x7f0634071c30 0x7f06340cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 >> v1:172.21.15.111:6789/0 conn(0x7f06341438f0 legacy=0x7f0634071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0624003b00 0x7f0624003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06240047c0 0x7f0624004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0624005180 0x7f062400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0634143770 con 0x7f0624005180 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062400c000 con 0x7f0624003b00 2023-12-04T05:39:44.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.416+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062400c1c0 con 0x7f06240047c0 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0624005180 0x7f062400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0638981640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0624003b00 0x7f0624003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0633fff640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06240047c0 0x7f0624004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 -- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06240047c0 msgr2=0x7f0624004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06240047c0 0x7f0624004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 -- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0624003b00 msgr2=0x7f0624003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0624003b00 0x7f0624003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 -- 172.21.15.111:0/3023739373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624003520 con 0x7f0624005180 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 -- 172.21.15.111:0/3023739373 reap_dead start 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.417+0000 7f0639182640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0624005180 0x7f062400b980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e320 tx=0x7f06240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c07040ab127454 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c02cde0 con 0x7f0624005180 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c00f040 con 0x7f0624005180 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f06337fe640 1 -- 172.21.15.111:0/3023739373 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c003bd0 con 0x7f0624005180 2023-12-04T05:39:44.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 -- 172.21.15.111:0/3023739373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0624005180 msgr2=0x7f062400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0624005180 0x7f062400b980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e320 tx=0x7f06240057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 -- 172.21.15.111:0/3023739373 shutdown_connections 2023-12-04T05:39:44.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 --2- 172.21.15.111:0/3023739373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0624005180 0x7f062400b980 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 -- 172.21.15.111:0/3023739373 >> 172.21.15.111:0/3023739373 conn(0x7f063406d4a0 msgr2=0x7f063406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:44.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 -- 172.21.15.111:0/3023739373 shutdown_connections 2023-12-04T05:39:44.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.418+0000 7f063abe5640 1 -- 172.21.15.111:0/3023739373 wait complete. 2023-12-04T05:39:44.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.419+0000 7f063abe5640 1 Processor -- start 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.419+0000 7f063abe5640 1 -- start start 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f063abe5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0624005180 0x7f06340b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f063abe5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06340ac870 0x7f06340accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f063abe5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 0x7f0634135870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f063abe5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340ad230 con 0x7f06341353f0 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f063abe5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340b12c0 con 0x7f06340ac870 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f063abe5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0634136120 con 0x7f0624005180 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0639182640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 0x7f0634135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0639182640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 0x7f0634135870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36666/0 (socket says 172.21.15.111:36666) 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0639182640 1 -- 172.21.15.111:0/1269159072 learned_addr learned my addr 172.21.15.111:0/1269159072 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0638981640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0624005180 0x7f06340b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0633fff640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06340ac870 0x7f06340accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0639182640 1 -- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0624005180 msgr2=0x7f06340b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.420+0000 7f0639182640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0624005180 0x7f06340b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f0639182640 1 -- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06340ac870 msgr2=0x7f06340accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f0639182640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06340ac870 0x7f06340accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f0639182640 1 -- 172.21.15.111:0/1269159072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634136400 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f0633fff640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06340ac870 0x7f06340accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f0639182640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 0x7f0634135870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e440 tx=0x7f062c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c004370 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0044d0 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c004880 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06341366e0 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.421+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634135db0 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.422+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0634069360 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.422+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f062c03f470 con 0x7f06341353f0 2023-12-04T05:39:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.422+0000 7f06317fa640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f061806b2a0 0x7f061806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.423+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f062c0ac740 con 0x7f06341353f0 2023-12-04T05:39:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.423+0000 7f0638981640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f061806b2a0 0x7f061806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.423+0000 7f0638981640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f061806b2a0 0x7f061806d760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0628000c00 tx=0x7f0628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.427+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f062c0775b0 con 0x7f06341353f0 2023-12-04T05:39:44.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.595+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0634069360 con 0x7f06341353f0 2023-12-04T05:39:44.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.600+0000 7f06317fa640 1 -- 172.21.15.111:0/1269159072 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f062c009de0 con 0x7f06341353f0 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f061806b2a0 msgr2=0x7f061806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f061806b2a0 0x7f061806d760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0628000c00 tx=0x7f0628009040 comp rx=0 tx=0).stop 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 msgr2=0x7f0634135870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 0x7f0634135870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e440 tx=0x7f062c009750 comp rx=0 tx=0).stop 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 shutdown_connections 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f061806b2a0 0x7f061806d760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06341353f0 0x7f0634135870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06340ac870 0x7f06340accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 --2- 172.21.15.111:0/1269159072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0624005180 0x7f06340b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 >> 172.21.15.111:0/1269159072 conn(0x7f063406d4a0 msgr2=0x7f0634071520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:44.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 shutdown_connections 2023-12-04T05:39:44.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.602+0000 7f063abe5640 1 -- 172.21.15.111:0/1269159072 wait complete. 2023-12-04T05:39:44.614 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-04T05:39:44.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:39:44.653 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:39:44.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-04T05:39:44.696 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:39:44.696 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:44.711 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:39:44.711 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:44.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.905+0000 7f4c6c844640 1 Processor -- start 2023-12-04T05:39:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.905+0000 7f4c6c844640 1 -- start start 2023-12-04T05:39:44.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.905+0000 7f4c6c844640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4c680cf9f0 0x7f4c680b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.905+0000 7f4c6c844640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680cbca0 con 0x7f4c6815b480 2023-12-04T05:39:44.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.905+0000 7f4c6c844640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680cbf40 con 0x7f4c6815f4e0 2023-12-04T05:39:44.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.905+0000 7f4c6c844640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680cc220 con 0x7f4c680cf9f0 2023-12-04T05:39:44.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4c6815b480 0x7f4c6815edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37892/0 (socket says 172.21.15.111:37892) 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 -- 172.21.15.111:0/2722895845 learned_addr learned my addr 172.21.15.111:0/2722895845 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c66d9d640 1 --2- 172.21.15.111:0/2722895845 >> v2:172.21.15.192:3300/0 conn(0x7f4c680cf9f0 0x7f4c680b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961315127 0 0) 0x7f4c680cbca0 con 0x7f4c6815b480 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c54003520 con 0x7f4c6815b480 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1881853820 0 0) 0x7f4c54003520 con 0x7f4c6815b480 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 >> v2:172.21.15.192:3300/0 conn(0x7f4c680cf9f0 msgr2=0x7f4c680b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 --2- 172.21.15.111:0/2722895845 >> v2:172.21.15.192:3300/0 conn(0x7f4c680cf9f0 0x7f4c680b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 >> v1:172.21.15.192:6789/0 conn(0x7f4c6815f4e0 legacy=0x7f4c680b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680cc500 con 0x7f4c6815b480 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f4c58001670 con 0x7f4c6815b480 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 >> v1:172.21.15.111:6789/0 conn(0x7f4c6815b480 legacy=0x7f4c6815edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c540047c0 0x7f4c54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54005180 0x7f4c5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680cc220 con 0x7f4c54003970 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c020 con 0x7f4c54005180 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c1e0 con 0x7f4c540047c0 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c66d9d640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54005180 0x7f4c5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c540047c0 0x7f4c54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 -- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c540047c0 msgr2=0x7f4c54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c540047c0 0x7f4c54004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 -- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54005180 msgr2=0x7f4c5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54005180 0x7f4c5400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c6659c640 1 -- 172.21.15.111:0/2722895845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54003520 con 0x7f4c54003970 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.906+0000 7f4c66d9d640 1 -- 172.21.15.111:0/2722895845 reap_dead start 2023-12-04T05:39:44.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.907+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c240 tx=0x7f4c540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871465777bd3d125 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.907+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58005ad0 con 0x7f4c54003970 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.907+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58005c30 con 0x7f4c54003970 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.907+0000 7f4c65d9b640 1 -- 172.21.15.111:0/2722895845 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5802cab0 con 0x7f4c54003970 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 -- 172.21.15.111:0/2722895845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 msgr2=0x7f4c54003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c240 tx=0x7f4c540057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 -- 172.21.15.111:0/2722895845 shutdown_connections 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2722895845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 -- 172.21.15.111:0/2722895845 >> 172.21.15.111:0/2722895845 conn(0x7f4c680b2a70 msgr2=0x7f4c680b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 -- 172.21.15.111:0/2722895845 shutdown_connections 2023-12-04T05:39:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.908+0000 7f4c6c844640 1 -- 172.21.15.111:0/2722895845 wait complete. 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.909+0000 7f4c6c844640 1 Processor -- start 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.909+0000 7f4c6c844640 1 -- start start 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6c844640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54003970 0x7f4c680af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6c844640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 0x7f4c6814f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6c844640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680c3d40 0x7f4c6814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6c844640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6814fcf0 con 0x7f4c680c38c0 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6c844640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680ca7f0 con 0x7f4c54003970 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6c844640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6806b770 con 0x7f4c680c3d40 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 0x7f4c6814f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 0x7f4c6814f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36690/0 (socket says 172.21.15.111:36690) 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 -- 172.21.15.111:0/2046731691 learned_addr learned my addr 172.21.15.111:0/2046731691 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c66d9d640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680c3d40 0x7f4c6814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 -- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680c3d40 msgr2=0x7f4c6814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54003970 0x7f4c680af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680c3d40 0x7f4c6814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 -- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54003970 msgr2=0x7f4c680af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54003970 0x7f4c680af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 -- 172.21.15.111:0/2046731691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6806ba50 con 0x7f4c680c38c0 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c66d9d640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680c3d40 0x7f4c6814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54003970 0x7f4c680af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.910+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 0x7f4c6814f270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e4d0 tx=0x7f4c5000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.911+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500052e0 con 0x7f4c680c38c0 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.911+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50005440 con 0x7f4c680c38c0 2023-12-04T05:39:44.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.911+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6806bd30 con 0x7f4c680c38c0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.911+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50003810 con 0x7f4c680c38c0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.911+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6806c2f0 con 0x7f4c680c38c0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.912+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6806c4b0 con 0x7f4c680c38c0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.912+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4c50003970 con 0x7f4c680c38c0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.912+0000 7f4c5effd640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.912+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c50089970 con 0x7f4c680c38c0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.912+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:44.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.913+0000 7f4c6659c640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c58034000 tx=0x7f4c58009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:44.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:44.917+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c500547e0 con 0x7f4c680c38c0 2023-12-04T05:39:45.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.085+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c68067460 con 0x7f4c680c38c0 2023-12-04T05:39:45.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.086+0000 7f4c5effd640 1 -- 172.21.15.111:0/2046731691 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f4c50057e00 con 0x7f4c680c38c0 2023-12-04T05:39:45.087 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:45.087 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:45.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 msgr2=0x7f4c4806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c58034000 tx=0x7f4c58009030 comp rx=0 tx=0).stop 2023-12-04T05:39:45.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 msgr2=0x7f4c6814f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 0x7f4c6814f270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e4d0 tx=0x7f4c5000e990 comp rx=0 tx=0).stop 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 shutdown_connections 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680c3d40 0x7f4c6814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c680c38c0 0x7f4c6814f270 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 --2- 172.21.15.111:0/2046731691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54003970 0x7f4c680af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 >> 172.21.15.111:0/2046731691 conn(0x7f4c680b2a70 msgr2=0x7f4c680b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 shutdown_connections 2023-12-04T05:39:45.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.088+0000 7f4c6c844640 1 -- 172.21.15.111:0/2046731691 wait complete. 2023-12-04T05:39:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:45.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:45.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.201+0000 7f694ae6b640 1 Processor -- start 2023-12-04T05:39:45.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.201+0000 7f694ae6b640 1 -- start start 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f694ae6b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f694415d480 0x7f69440d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f694ae6b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69440d1310 0x7f69440cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f694ae6b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694415ddc0 con 0x7f694415d860 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f694ae6b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694415e0a0 con 0x7f694415d480 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f694ae6b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694415e380 con 0x7f69440d1310 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f6948c07640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f694415d480 0x7f69440d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f6948c07640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f694415d480 0x7f69440d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36694/0 (socket says 172.21.15.111:36694) 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.202+0000 7f6948c07640 1 -- 172.21.15.111:0/2484489775 learned_addr learned my addr 172.21.15.111:0/2484489775 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.203+0000 7f6943fff640 1 --2- 172.21.15.111:0/2484489775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69440d1310 0x7f69440cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.203+0000 7f6948c07640 1 -- 172.21.15.111:0/2484489775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69440d1310 msgr2=0x7f69440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.203+0000 7f6948c07640 1 --2- 172.21.15.111:0/2484489775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69440d1310 0x7f69440cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.203+0000 7f6948c07640 1 -- 172.21.15.111:0/2484489775 >> v1:172.21.15.192:6789/0 conn(0x7f694415d860 legacy=0x7f6944160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.203+0000 7f6948c07640 1 -- 172.21.15.111:0/2484489775 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694415e660 con 0x7f694415d480 2023-12-04T05:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.203+0000 7f6948c07640 1 --2- 172.21.15.111:0/2484489775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f694415d480 0x7f69440d0dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6934009870 tx=0x7f693402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12cb57fd7d05e0d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f69437fe640 1 -- 172.21.15.111:0/2484489775 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934031640 con 0x7f694415d480 2023-12-04T05:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f69437fe640 1 -- 172.21.15.111:0/2484489775 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69340317a0 con 0x7f694415d480 2023-12-04T05:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f69437fe640 1 -- 172.21.15.111:0/2484489775 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934031b30 con 0x7f694415d480 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 -- 172.21.15.111:0/2484489775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f694415d480 msgr2=0x7f69440d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 --2- 172.21.15.111:0/2484489775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f694415d480 0x7f69440d0dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6934009870 tx=0x7f693402b8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 -- 172.21.15.111:0/2484489775 shutdown_connections 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 --2- 172.21.15.111:0/2484489775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69440d1310 0x7f69440cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 --2- 172.21.15.111:0/2484489775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f694415d480 0x7f69440d0dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 -- 172.21.15.111:0/2484489775 >> 172.21.15.111:0/2484489775 conn(0x7f694409c540 msgr2=0x7f69440d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 -- 172.21.15.111:0/2484489775 shutdown_connections 2023-12-04T05:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.204+0000 7f694ae6b640 1 -- 172.21.15.111:0/2484489775 wait complete. 2023-12-04T05:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.205+0000 7f694ae6b640 1 Processor -- start 2023-12-04T05:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.205+0000 7f694ae6b640 1 -- start start 2023-12-04T05:39:45.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f694ae6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 0x7f694415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f694ae6b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f694415d480 0x7f69440afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f694ae6b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f694415d860 0x7f69440b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f694ae6b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694415ad30 con 0x7f69440d1310 2023-12-04T05:39:45.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f694ae6b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b3c70 con 0x7f694415d480 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f694ae6b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440cbf00 con 0x7f694415d860 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f6948c07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 0x7f694415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f6948c07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 0x7f694415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36706/0 (socket says 172.21.15.111:36706) 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.206+0000 7f6948c07640 1 -- 172.21.15.111:0/4094648574 learned_addr learned my addr 172.21.15.111:0/4094648574 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6943fff640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f694415d480 0x7f69440afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6949408640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f694415d860 0x7f69440b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6948c07640 1 -- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f694415d860 msgr2=0x7f69440b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6948c07640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f694415d860 0x7f69440b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6948c07640 1 -- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f694415d480 msgr2=0x7f69440afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6948c07640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f694415d480 0x7f69440afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6948c07640 1 -- 172.21.15.111:0/4094648574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340094d0 con 0x7f69440d1310 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6943fff640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f694415d480 0x7f69440afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:45.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6949408640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f694415d860 0x7f69440b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f6948c07640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 0x7f694415a7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6934009870 tx=0x7f69340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693403ac60 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693403ade0 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69440cc1e0 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934044630 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.207+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69440b0370 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.208+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6944067960 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.208+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f693402f050 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.209+0000 7f69417fa640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f693006b2a0 0x7f693006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.209+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69340abcd0 con 0x7f69440d1310 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.209+0000 7f6943fff640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f693006b2a0 0x7f693006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.209+0000 7f6943fff640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f693006b2a0 0x7f693006d760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69440b2ba0 tx=0x7f6938008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.213+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6934076bd0 con 0x7f69440d1310 2023-12-04T05:39:45.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.381+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69440cbd40 con 0x7f69440d1310 2023-12-04T05:39:45.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.381+0000 7f69417fa640 1 -- 172.21.15.111:0/4094648574 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6934035020 con 0x7f69440d1310 2023-12-04T05:39:45.383 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:45.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.383+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f693006b2a0 msgr2=0x7f693006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.383+0000 7f694ae6b640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f693006b2a0 0x7f693006d760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69440b2ba0 tx=0x7f6938008040 comp rx=0 tx=0).stop 2023-12-04T05:39:45.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.383+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 msgr2=0x7f694415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.383+0000 7f694ae6b640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 0x7f694415a7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6934009870 tx=0x7f69340046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 shutdown_connections 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f693006b2a0 0x7f693006d760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f694415d860 0x7f69440b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f694415d480 0x7f69440afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 --2- 172.21.15.111:0/4094648574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69440d1310 0x7f694415a7f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 >> 172.21.15.111:0/4094648574 conn(0x7f694409c540 msgr2=0x7f69440b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 shutdown_connections 2023-12-04T05:39:45.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.384+0000 7f694ae6b640 1 -- 172.21.15.111:0/4094648574 wait complete. 2023-12-04T05:39:45.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:45.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.497+0000 7f8e17dd5640 1 Processor -- start 2023-12-04T05:39:45.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.497+0000 7f8e17dd5640 1 -- start start 2023-12-04T05:39:45.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e17dd5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8e1013cee0 0x7f8e1013d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e17dd5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8e1013f640 0x7f8e1013fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e17dd5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e10072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e17dd5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10072670 con 0x7f8e1013cee0 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e17dd5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1013d2c0 con 0x7f8e1013f640 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e17dd5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101400e0 con 0x7f8e10071d20 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8e1013cee0 0x7f8e1013d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8e1013cee0 0x7f8e1013d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36722/0 (socket says 172.21.15.111:36722) 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 -- 172.21.15.111:0/653710542 learned_addr learned my addr 172.21.15.111:0/653710542 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 -- 172.21.15.111:0/653710542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 msgr2=0x7f8e10072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 --2- 172.21.15.111:0/653710542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e10072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 -- 172.21.15.111:0/653710542 >> v2:172.21.15.192:3300/0 conn(0x7f8e1013f640 msgr2=0x7f8e1013fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 --2- 172.21.15.111:0/653710542 >> v2:172.21.15.192:3300/0 conn(0x7f8e1013f640 0x7f8e1013fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.498+0000 7f8e15b71640 1 -- 172.21.15.111:0/653710542 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e101403c0 con 0x7f8e1013cee0 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e15b71640 1 --2- 172.21.15.111:0/653710542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013cee0 0x7f8e1013d600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8e10143c70 tx=0x7f8e0c02dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfa87b6ddde404d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e14b6f640 1 -- 172.21.15.111:0/653710542 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c02ed70 con 0x7f8e1013cee0 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e14b6f640 1 -- 172.21.15.111:0/653710542 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c036440 con 0x7f8e1013cee0 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e14b6f640 1 -- 172.21.15.111:0/653710542 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0367d0 con 0x7f8e1013cee0 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 -- 172.21.15.111:0/653710542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013cee0 msgr2=0x7f8e1013d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/653710542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013cee0 0x7f8e1013d600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8e10143c70 tx=0x7f8e0c02dfc0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 -- 172.21.15.111:0/653710542 shutdown_connections 2023-12-04T05:39:45.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/653710542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e10072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/653710542 >> v2:172.21.15.192:3300/0 conn(0x7f8e1013f640 0x7f8e1013fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/653710542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013cee0 0x7f8e1013d600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.499+0000 7f8e17dd5640 1 -- 172.21.15.111:0/653710542 >> 172.21.15.111:0/653710542 conn(0x7f8e100b0cf0 msgr2=0x7f8e100b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.500+0000 7f8e17dd5640 1 -- 172.21.15.111:0/653710542 shutdown_connections 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.500+0000 7f8e17dd5640 1 -- 172.21.15.111:0/653710542 wait complete. 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 Processor -- start 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 -- start start 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e1013c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e1013cee0 0x7f8e100a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 0x7f8e100b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10152b60 con 0x7f8e1013f640 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10152e40 con 0x7f8e1013cee0 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.501+0000 7f8e17dd5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10153120 con 0x7f8e10071d20 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 0x7f8e100b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 0x7f8e100b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36724/0 (socket says 172.21.15.111:36724) 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 -- 172.21.15.111:0/2049976038 learned_addr learned my addr 172.21.15.111:0/2049976038 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e15b71640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e1013c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e15370640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e1013cee0 0x7f8e100a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 -- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 msgr2=0x7f8e1013c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e1013c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 -- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e1013cee0 msgr2=0x7f8e100a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e1013cee0 0x7f8e100a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 -- 172.21.15.111:0/2049976038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c00b770 con 0x7f8e1013f640 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e15370640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e1013cee0 0x7f8e100a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e16372640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 0x7f8e100b38e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800b970 tx=0x7f8e0800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800cb40 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0800ccc0 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e10153400 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08014680 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.502+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1013c630 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.503+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e10153920 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.504+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8e0801d3b0 con 0x7f8e1013f640 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.504+0000 7f8e067fc640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8df406b2a0 0x7f8df406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.504+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e08089a10 con 0x7f8e1013f640 2023-12-04T05:39:45.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.504+0000 7f8e15b71640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8df406b2a0 0x7f8df406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.504+0000 7f8e15b71640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8df406b2a0 0x7f8df406d760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c038000 tx=0x7f8e0c032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.508+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e10153920 con 0x7f8e1013f640 2023-12-04T05:39:45.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.674+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e10153920 con 0x7f8e1013f640 2023-12-04T05:39:45.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.674+0000 7f8e067fc640 1 -- 172.21.15.111:0/2049976038 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e10153920 con 0x7f8e1013f640 2023-12-04T05:39:45.676 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8df406b2a0 msgr2=0x7f8df406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8df406b2a0 0x7f8df406d760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c038000 tx=0x7f8e0c032000 comp rx=0 tx=0).stop 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 msgr2=0x7f8e100b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 0x7f8e100b38e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800b970 tx=0x7f8e0800be30 comp rx=0 tx=0).stop 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 shutdown_connections 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8df406b2a0 0x7f8df406d760 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e1013f640 0x7f8e100b38e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e1013cee0 0x7f8e100a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 --2- 172.21.15.111:0/2049976038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10071d20 0x7f8e1013c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 >> 172.21.15.111:0/2049976038 conn(0x7f8e100b0cf0 msgr2=0x7f8e10158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 shutdown_connections 2023-12-04T05:39:45.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.676+0000 7f8e17dd5640 1 -- 172.21.15.111:0/2049976038 wait complete. 2023-12-04T05:39:45.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:45.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.790+0000 7f666b7a5640 1 Processor -- start 2023-12-04T05:39:45.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.790+0000 7f666b7a5640 1 -- start start 2023-12-04T05:39:45.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f666b7a5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f66640d1620 0x7f6664161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f666b7a5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666406eaf0 con 0x7f666415d8a0 2023-12-04T05:39:45.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f666b7a5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666415e4e0 con 0x7f6664146b10 2023-12-04T05:39:45.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f666b7a5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666415e7c0 con 0x7f66640d1620 2023-12-04T05:39:45.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f6669d42640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f666415d8a0 0x7f666415dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37908/0 (socket says 172.21.15.111:37908) 2023-12-04T05:39:45.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f6669d42640 1 -- 172.21.15.111:0/1044775810 learned_addr learned my addr 172.21.15.111:0/1044775810 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:45.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f6668d40640 1 --2- 172.21.15.111:0/1044775810 >> v2:172.21.15.192:3300/0 conn(0x7f66640d1620 0x7f6664161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f665bfff640 1 -- 172.21.15.111:0/1044775810 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375099672 0 0) 0x7f666406eaf0 con 0x7f666415d8a0 2023-12-04T05:39:45.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f665bfff640 1 -- 172.21.15.111:0/1044775810 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6650003520 con 0x7f666415d8a0 2023-12-04T05:39:45.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f6668d40640 1 -- 172.21.15.111:0/1044775810 >> v1:172.21.15.192:6789/0 conn(0x7f6664146b10 legacy=0x7f66640d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f6668d40640 1 -- 172.21.15.111:0/1044775810 >> v1:172.21.15.111:6789/0 conn(0x7f666415d8a0 legacy=0x7f666415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.791+0000 7f6668d40640 1 -- 172.21.15.111:0/1044775810 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664160f50 con 0x7f66640d1620 2023-12-04T05:39:45.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.792+0000 7f6668d40640 1 --2- 172.21.15.111:0/1044775810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66640d1620 0x7f6664161a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66540097a0 tx=0x7f665402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f64708978a04cb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.792+0000 7f665bfff640 1 -- 172.21.15.111:0/1044775810 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654030610 con 0x7f66640d1620 2023-12-04T05:39:45.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.792+0000 7f665bfff640 1 -- 172.21.15.111:0/1044775810 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654030770 con 0x7f66640d1620 2023-12-04T05:39:45.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.792+0000 7f665bfff640 1 -- 172.21.15.111:0/1044775810 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654030ae0 con 0x7f66640d1620 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 -- 172.21.15.111:0/1044775810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66640d1620 msgr2=0x7f6664161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 --2- 172.21.15.111:0/1044775810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66640d1620 0x7f6664161a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66540097a0 tx=0x7f665402b960 comp rx=0 tx=0).stop 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 -- 172.21.15.111:0/1044775810 shutdown_connections 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 --2- 172.21.15.111:0/1044775810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66640d1620 0x7f6664161a30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 -- 172.21.15.111:0/1044775810 >> 172.21.15.111:0/1044775810 conn(0x7f6664070af0 msgr2=0x7f66640662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 -- 172.21.15.111:0/1044775810 shutdown_connections 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.793+0000 7f666b7a5640 1 -- 172.21.15.111:0/1044775810 wait complete. 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.794+0000 7f666b7a5640 1 Processor -- start 2023-12-04T05:39:45.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.794+0000 7f666b7a5640 1 -- start start 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66640d1620 0x7f66640ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 0x7f666413b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666415d8a0 0x7f666413f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666415d140 con 0x7f6664146b10 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666413d220 con 0x7f666415d8a0 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66641410b0 con 0x7f66640d1620 2023-12-04T05:39:45.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 0x7f666413b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 0x7f666413b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36726/0 (socket says 172.21.15.111:36726) 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 -- 172.21.15.111:0/2582449987 learned_addr learned my addr 172.21.15.111:0/2582449987 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6669d42640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666415d8a0 0x7f666413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6669541640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66640d1620 0x7f66640ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 -- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66640d1620 msgr2=0x7f66640ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66640d1620 0x7f66640ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 -- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666415d8a0 msgr2=0x7f666413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666415d8a0 0x7f666413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 -- 172.21.15.111:0/2582449987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009400 con 0x7f6664146b10 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6669d42640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666415d8a0 0x7f666413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6668d40640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 0x7f666413b2e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6654000c00 tx=0x7f66540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654039c90 con 0x7f6664146b10 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665403f040 con 0x7f6664146b10 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66640caa70 con 0x7f6664146b10 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.795+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654006e10 con 0x7f6664146b10 2023-12-04T05:39:45.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.796+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666413cbd0 con 0x7f6664146b10 2023-12-04T05:39:45.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.797+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666413d390 con 0x7f6664146b10 2023-12-04T05:39:45.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.797+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6654004030 con 0x7f6664146b10 2023-12-04T05:39:45.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.797+0000 7f6659ffb640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f664806b2a0 0x7f664806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:45.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.798+0000 7f6669541640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f664806b2a0 0x7f664806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:45.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.798+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66540abbd0 con 0x7f6664146b10 2023-12-04T05:39:45.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.798+0000 7f6669541640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f664806b2a0 0x7f664806d760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6660001640 tx=0x7f6660009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:45.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.802+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6654076a40 con 0x7f6664146b10 2023-12-04T05:39:45.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.968+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6664069360 con 0x7f6664146b10 2023-12-04T05:39:45.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.969+0000 7f6659ffb640 1 -- 172.21.15.111:0/2582449987 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6654048020 con 0x7f6664146b10 2023-12-04T05:39:45.970 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.970+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f664806b2a0 msgr2=0x7f664806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.970+0000 7f666b7a5640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f664806b2a0 0x7f664806d760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6660001640 tx=0x7f6660009040 comp rx=0 tx=0).stop 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.970+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 msgr2=0x7f666413b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.970+0000 7f666b7a5640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 0x7f666413b2e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6654000c00 tx=0x7f66540046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 shutdown_connections 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f664806b2a0 0x7f664806d760 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f666415d8a0 0x7f666413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6664146b10 0x7f666413b2e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 --2- 172.21.15.111:0/2582449987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66640d1620 0x7f66640ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:45.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 >> 172.21.15.111:0/2582449987 conn(0x7f6664070af0 msgr2=0x7f66640b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:45.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 shutdown_connections 2023-12-04T05:39:45.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:45.971+0000 7f666b7a5640 1 -- 172.21.15.111:0/2582449987 wait complete. 2023-12-04T05:39:45.983 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:46.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.082+0000 7f45052ee640 1 Processor -- start 2023-12-04T05:39:46.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.082+0000 7f45052ee640 1 -- start start 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f45052ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f450015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f45052ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4500140200 0x7f450015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f45052ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45000aea20 con 0x7f450013cee0 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f45052ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450013d2c0 con 0x7f4500140200 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f45052ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450015b3a0 con 0x7f450006c960 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44ff7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4500140200 0x7f450015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44feffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f450013cee0 0x7f450006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37924/0 (socket says 172.21.15.111:37924) 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44feffd640 1 -- 172.21.15.111:0/3784404800 learned_addr learned my addr 172.21.15.111:0/3784404800 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44fdffb640 1 -- 172.21.15.111:0/3784404800 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444610915 0 0) 0x7f45000aea20 con 0x7f450013cee0 2023-12-04T05:39:46.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44fdffb640 1 -- 172.21.15.111:0/3784404800 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8003520 con 0x7f450013cee0 2023-12-04T05:39:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44fe7fc640 1 --2- 172.21.15.111:0/3784404800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f450015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44ff7fe640 1 -- 172.21.15.111:0/3784404800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 msgr2=0x7f450015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44ff7fe640 1 --2- 172.21.15.111:0/3784404800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f450015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44ff7fe640 1 -- 172.21.15.111:0/3784404800 >> v1:172.21.15.111:6789/0 conn(0x7f450013cee0 legacy=0x7f450006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.083+0000 7f44ff7fe640 1 -- 172.21.15.111:0/3784404800 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450015b0c0 con 0x7f4500140200 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.084+0000 7f44ff7fe640 1 --2- 172.21.15.111:0/3784404800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500140200 0x7f450015a9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097a0 tx=0x7f44f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d41acb0505afea7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.084+0000 7f44fdffb640 1 -- 172.21.15.111:0/3784404800 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f802e070 con 0x7f4500140200 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.084+0000 7f44fdffb640 1 -- 172.21.15.111:0/3784404800 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f80315a0 con 0x7f4500140200 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.084+0000 7f44fdffb640 1 -- 172.21.15.111:0/3784404800 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8031930 con 0x7f4500140200 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 -- 172.21.15.111:0/3784404800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500140200 msgr2=0x7f450015a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 --2- 172.21.15.111:0/3784404800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500140200 0x7f450015a9d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097a0 tx=0x7f44f802b840 comp rx=0 tx=0).stop 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 -- 172.21.15.111:0/3784404800 shutdown_connections 2023-12-04T05:39:46.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 --2- 172.21.15.111:0/3784404800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4500140200 0x7f450015a9d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 --2- 172.21.15.111:0/3784404800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f450015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 -- 172.21.15.111:0/3784404800 >> 172.21.15.111:0/3784404800 conn(0x7f450006a4c0 msgr2=0x7f450006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:46.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 -- 172.21.15.111:0/3784404800 shutdown_connections 2023-12-04T05:39:46.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.085+0000 7f45052ee640 1 -- 172.21.15.111:0/3784404800 wait complete. 2023-12-04T05:39:46.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.086+0000 7f45052ee640 1 Processor -- start 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.086+0000 7f45052ee640 1 -- start start 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f45052ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f4500150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f45052ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 0x7f450015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f45052ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4500140200 0x7f450015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f45052ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450013f150 con 0x7f450013cee0 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f45052ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45001f37c0 con 0x7f4500140200 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f45052ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45000ac910 con 0x7f450006c960 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 0x7f450015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 0x7f450015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36734/0 (socket says 172.21.15.111:36734) 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 -- 172.21.15.111:0/328922385 learned_addr learned my addr 172.21.15.111:0/328922385 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44feffd640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f4500150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44ff7fe640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4500140200 0x7f450015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 -- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 msgr2=0x7f4500150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f4500150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 -- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4500140200 msgr2=0x7f450015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4500140200 0x7f450015e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.087+0000 7f44fe7fc640 1 -- 172.21.15.111:0/328922385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8009400 con 0x7f450013cee0 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.088+0000 7f44fe7fc640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 0x7f450015dfb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f44f400d800 tx=0x7f44f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.088+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400b6c0 con 0x7f450013cee0 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.088+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f400b820 con 0x7f450013cee0 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.088+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45000acbf0 con 0x7f450013cee0 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.088+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400bbb0 con 0x7f450013cee0 2023-12-04T05:39:46.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.088+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4500150550 con 0x7f450013cee0 2023-12-04T05:39:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.089+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45000693d0 con 0x7f450013cee0 2023-12-04T05:39:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.089+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f44f400bd10 con 0x7f450013cee0 2023-12-04T05:39:46.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.089+0000 7f44e77fe640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f44ec06b2a0 0x7f44ec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.090+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44f4089ff0 con 0x7f450013cee0 2023-12-04T05:39:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.090+0000 7f44feffd640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f44ec06b2a0 0x7f44ec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.090+0000 7f44feffd640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f44ec06b2a0 0x7f44ec06d760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001560 tx=0x7f44f0001a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.094+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44f4054f10 con 0x7f450013cee0 2023-12-04T05:39:46.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.273+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f450006d170 con 0x7f450013cee0 2023-12-04T05:39:46.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.273+0000 7f44e77fe640 1 -- 172.21.15.111:0/328922385 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44f4058530 con 0x7f450013cee0 2023-12-04T05:39:46.275 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f44ec06b2a0 msgr2=0x7f44ec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f44ec06b2a0 0x7f44ec06d760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001560 tx=0x7f44f0001a90 comp rx=0 tx=0).stop 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 msgr2=0x7f450015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 0x7f450015dfb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f44f400d800 tx=0x7f44f400dcc0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 shutdown_connections 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f44ec06b2a0 0x7f44ec06d760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4500140200 0x7f450015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f450013cee0 0x7f450015dfb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 --2- 172.21.15.111:0/328922385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f450006c960 0x7f4500150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 >> 172.21.15.111:0/328922385 conn(0x7f450006a4c0 msgr2=0x7f45000afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:46.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 shutdown_connections 2023-12-04T05:39:46.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.275+0000 7f45052ee640 1 -- 172.21.15.111:0/328922385 wait complete. 2023-12-04T05:39:46.288 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:39:46.288 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:46.291 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:39:46.304 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:46.304 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.442+0000 7fcdfb432640 1 Processor -- start 2023-12-04T05:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.442+0000 7fcdfb432640 1 -- start start 2023-12-04T05:39:46.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdfb432640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 0x7fcdf406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdfb432640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40aea20 con 0x7fcdf415aba0 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdfb432640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf413d2c0 con 0x7fcdf4140200 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdfb432640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415b3a0 con 0x7fcdf413cee0 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdf99cf640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcdf415aba0 0x7fcdf40cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37934/0 (socket says 172.21.15.111:37934) 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdf99cf640 1 -- 172.21.15.111:0/675783710 learned_addr learned my addr 172.21.15.111:0/675783710 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339812509 0 0) 0x7fcdf40aea20 con 0x7fcdf415aba0 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 0x7fcdf406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde0003520 con 0x7fcdf415aba0 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 736356300 0 0) 0x7fcde0003520 con 0x7fcdf415aba0 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 msgr2=0x7fcdf406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 0x7fcdf406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 >> v1:172.21.15.192:6789/0 conn(0x7fcdf4140200 legacy=0x7fcdf415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf415b0c0 con 0x7fcdf415aba0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fcdec0019f0 con 0x7fcdf415aba0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 >> v1:172.21.15.111:6789/0 conn(0x7fcdf415aba0 legacy=0x7fcdf40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.443+0000 7fcdebfff640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcde0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdebfff640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 0x7fcde00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdebfff640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde0004d00 0x7fcde0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415b3a0 con 0x7fcde00039a0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde000bab0 con 0x7fcde0004d00 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde000bc70 con 0x7fcdf413cee0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcde0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf89cd640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 0x7fcde00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf99cf640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde0004d00 0x7fcde0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf91ce640 1 -- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 msgr2=0x7fcde00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf413cee0 0x7fcde00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf91ce640 1 -- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde0004d00 msgr2=0x7fcde0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:46.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde0004d00 0x7fcde0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.444+0000 7fcdf91ce640 1 -- 172.21.15.111:0/675783710 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0003520 con 0x7fcde00039a0 2023-12-04T05:39:46.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcde0003dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009ac0 tx=0x7fcde00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704cd339c64d7834 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf002c600 con 0x7fcde00039a0 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf002c760 con 0x7fcde00039a0 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdebfff640 1 -- 172.21.15.111:0/675783710 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf002caf0 con 0x7fcde00039a0 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdfb432640 1 -- 172.21.15.111:0/675783710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 msgr2=0x7fcde0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdfb432640 1 --2- 172.21.15.111:0/675783710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcde0003dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009ac0 tx=0x7fcde00057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdf99cf640 1 -- 172.21.15.111:0/675783710 reap_dead start 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdfb432640 1 -- 172.21.15.111:0/675783710 shutdown_connections 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdfb432640 1 -- 172.21.15.111:0/675783710 >> 172.21.15.111:0/675783710 conn(0x7fcdf406a4c0 msgr2=0x7fcdf406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdfb432640 1 -- 172.21.15.111:0/675783710 shutdown_connections 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.445+0000 7fcdfb432640 1 -- 172.21.15.111:0/675783710 wait complete. 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 Processor -- start 2023-12-04T05:39:46.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 -- start start 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcdf41380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf40aec10 0x7fcdf41e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf4071a80 0x7fcdf41e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4138a00 con 0x7fcde00039a0 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40af110 con 0x7fcdf4071a80 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdfb432640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4070fa0 con 0x7fcdf40aec10 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdf91ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcdf41380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdf91ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcdf41380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36742/0 (socket says 172.21.15.111:36742) 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.447+0000 7fcdf91ce640 1 -- 172.21.15.111:0/2947747034 learned_addr learned my addr 172.21.15.111:0/2947747034 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf99cf640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf4071a80 0x7fcdf41e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf89cd640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf40aec10 0x7fcdf41e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf91ce640 1 -- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf40aec10 msgr2=0x7fcdf41e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf40aec10 0x7fcdf41e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf91ce640 1 -- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf4071a80 msgr2=0x7fcdf41e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf4071a80 0x7fcdf41e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf91ce640 1 -- 172.21.15.111:0/2947747034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4071280 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf89cd640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf40aec10 0x7fcdf41e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdf91ce640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcdf41380b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009ac0 tx=0x7fcdf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003810 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0003970 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf4071560 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003d00 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.448+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf41f32f0 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.449+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4070e00 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.450+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fcdf0040450 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.450+0000 7fcde9ffb640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcdd806b2d0 0x7fcdd806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.450+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdf00ab900 con 0x7fcde00039a0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.450+0000 7fcdf89cd640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcdd806b2d0 0x7fcdd806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.451+0000 7fcdf89cd640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcdd806b2d0 0x7fcdd806d790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdf406e730 tx=0x7fcde4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.454+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdf4070e00 con 0x7fcde00039a0 2023-12-04T05:39:46.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.623+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdf4070e00 con 0x7fcde00039a0 2023-12-04T05:39:46.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.624+0000 7fcde9ffb640 1 -- 172.21.15.111:0/2947747034 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fcdf4070e00 con 0x7fcde00039a0 2023-12-04T05:39:46.625 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:46.625 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcdd806b2d0 msgr2=0x7fcdd806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcdd806b2d0 0x7fcdd806d790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdf406e730 tx=0x7fcde4009040 comp rx=0 tx=0).stop 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 msgr2=0x7fcdf41380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcdf41380b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009ac0 tx=0x7fcdf0004730 comp rx=0 tx=0).stop 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 shutdown_connections 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcdd806b2d0 0x7fcdd806d790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf4071a80 0x7fcdf41e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf40aec10 0x7fcdf41e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 --2- 172.21.15.111:0/2947747034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcde00039a0 0x7fcdf41380b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 >> 172.21.15.111:0/2947747034 conn(0x7fcdf406a4c0 msgr2=0x7fcdf40adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:46.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 shutdown_connections 2023-12-04T05:39:46.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.626+0000 7fcdfb432640 1 -- 172.21.15.111:0/2947747034 wait complete. 2023-12-04T05:39:46.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:46.640 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:46.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.741+0000 7f3caea09640 1 Processor -- start 2023-12-04T05:39:46.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.741+0000 7f3caea09640 1 -- start start 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3caea09640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3caea09640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3ca806e4d0 0x7f3ca806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3caea09640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca815fbb0 con 0x7f3ca806ba10 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3caea09640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca815c6c0 con 0x7f3ca806e4d0 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3caea09640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca80cec20 con 0x7f3ca8158da0 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3ca77fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3ca806e4d0 0x7f3ca806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3cacfa6640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f3ca806ba10 0x7f3ca806c9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:37636/0 (socket says 172.21.15.111:37636) 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3cacfa6640 1 -- 172.21.15.111:0/3825416412 learned_addr learned my addr 172.21.15.111:0/3825416412 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3ca77fe640 1 -- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 msgr2=0x7f3ca815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3ca77fe640 1 --2- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3ca77fe640 1 -- 172.21.15.111:0/3825416412 >> v1:172.21.15.192:6789/0 conn(0x7f3ca806ba10 legacy=0x7f3ca806c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.742+0000 7f3ca77fe640 1 -- 172.21.15.111:0/3825416412 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca815edf0 con 0x7f3ca806e4d0 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.743+0000 7f3ca77fe640 1 --2- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca806b4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0097a0 tx=0x7f3c9c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34333e47cfaf91a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.743+0000 7f3ca6ffd640 1 -- 172.21.15.111:0/3825416412 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c031830 con 0x7f3ca806e4d0 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.743+0000 7f3ca6ffd640 1 -- 172.21.15.111:0/3825416412 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c031990 con 0x7f3ca806e4d0 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.743+0000 7f3ca6ffd640 1 -- 172.21.15.111:0/3825416412 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c031d20 con 0x7f3ca806e4d0 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 -- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 msgr2=0x7f3ca806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 --2- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca806b4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0097a0 tx=0x7f3c9c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 -- 172.21.15.111:0/3825416412 shutdown_connections 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 --2- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca806b4d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 --2- 172.21.15.111:0/3825416412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 -- 172.21.15.111:0/3825416412 >> 172.21.15.111:0/3825416412 conn(0x7f3ca80b1b90 msgr2=0x7f3ca8070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 -- 172.21.15.111:0/3825416412 shutdown_connections 2023-12-04T05:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.744+0000 7f3caea09640 1 -- 172.21.15.111:0/3825416412 wait complete. 2023-12-04T05:39:46.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.745+0000 7f3caea09640 1 Processor -- start 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.745+0000 7f3caea09640 1 -- start start 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3caea09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 0x7f3ca81526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3caea09640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca80ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3caea09640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3caea09640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca813f7a0 con 0x7f3ca806ba10 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3caea09640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca80a9850 con 0x7f3ca806e4d0 2023-12-04T05:39:46.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3caea09640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca813ee70 con 0x7f3ca8158da0 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 0x7f3ca81526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 0x7f3ca81526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36752/0 (socket says 172.21.15.111:36752) 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 -- 172.21.15.111:0/2593665103 learned_addr learned my addr 172.21.15.111:0/2593665103 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3cacfa6640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca77fe640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca80ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 -- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 msgr2=0x7f3ca815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 -- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 msgr2=0x7f3ca80ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca80ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca7fff640 1 -- 172.21.15.111:0/2593665103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c009400 con 0x7f3ca806ba10 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3cacfa6640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.746+0000 7f3ca77fe640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca80ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.747+0000 7f3ca7fff640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 0x7f3ca81526d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e8a0 tx=0x7f3c9800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.747+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c980094f0 con 0x7f3ca806ba10 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.747+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca8152c10 con 0x7f3ca806ba10 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.747+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800c940 con 0x7f3ca806ba10 2023-12-04T05:39:46.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.747+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800ccf0 con 0x7f3ca806ba10 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.747+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca8141cd0 con 0x7f3ca806ba10 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.748+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca8069360 con 0x7f3ca806ba10 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.748+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3c98013470 con 0x7f3ca806ba10 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.749+0000 7f3ca4ff9640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3c9406b2a0 0x7f3c9406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.749+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c98089c60 con 0x7f3ca806ba10 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.749+0000 7f3ca77fe640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3c9406b2a0 0x7f3c9406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:46.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.749+0000 7f3ca77fe640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3c9406b2a0 0x7f3c9406d760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c036000 tx=0x7f3c9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:46.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.753+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c98054ae0 con 0x7f3ca806ba10 2023-12-04T05:39:46.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.920+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ca8071ad0 con 0x7f3ca806ba10 2023-12-04T05:39:46.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.921+0000 7f3ca4ff9640 1 -- 172.21.15.111:0/2593665103 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c98058100 con 0x7f3ca806ba10 2023-12-04T05:39:46.922 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3c9406b2a0 msgr2=0x7f3c9406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3c9406b2a0 0x7f3c9406d760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c036000 tx=0x7f3c9c008030 comp rx=0 tx=0).stop 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 msgr2=0x7f3ca81526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 0x7f3ca81526d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e8a0 tx=0x7f3c9800ed60 comp rx=0 tx=0).stop 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 shutdown_connections 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3c9406b2a0 0x7f3c9406d760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ca8158da0 0x7f3ca815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ca806e4d0 0x7f3ca80ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 --2- 172.21.15.111:0/2593665103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ca806ba10 0x7f3ca81526d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:46.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.922+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 >> 172.21.15.111:0/2593665103 conn(0x7f3ca80b1b90 msgr2=0x7f3ca8152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:46.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.923+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 shutdown_connections 2023-12-04T05:39:46.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:46.923+0000 7f3caea09640 1 -- 172.21.15.111:0/2593665103 wait complete. 2023-12-04T05:39:46.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:47.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.037+0000 7f6e9e105640 1 Processor -- start 2023-12-04T05:39:47.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.037+0000 7f6e9e105640 1 -- start start 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e9e105640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e980d1780 0x7f6e980d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e9e105640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b2ad0 0x7f6e980ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e9e105640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e980cdcb0 con 0x7f6e980b20a0 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e9e105640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e980d39e0 con 0x7f6e980d1780 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e9e105640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980d3cc0 con 0x7f6e980b2ad0 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e980d1780 0x7f6e980d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e980d1780 0x7f6e980d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36766/0 (socket says 172.21.15.111:36766) 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 -- 172.21.15.111:0/1536594091 learned_addr learned my addr 172.21.15.111:0/1536594091 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e97fff640 1 --2- 172.21.15.111:0/1536594091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b2ad0 0x7f6e980ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 -- 172.21.15.111:0/1536594091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b2ad0 msgr2=0x7f6e980ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:47.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 --2- 172.21.15.111:0/1536594091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b2ad0 0x7f6e980ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 -- 172.21.15.111:0/1536594091 >> v1:172.21.15.192:6789/0 conn(0x7f6e980b20a0 legacy=0x7f6e980d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.038+0000 7f6e977fe640 1 -- 172.21.15.111:0/1536594091 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980cdf50 con 0x7f6e980d1780 2023-12-04T05:39:47.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.039+0000 7f6e977fe640 1 --2- 172.21.15.111:0/1536594091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980d01e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009870 tx=0x7f6e8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44ec42b2d2a1fb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.039+0000 7f6e967fc640 1 -- 172.21.15.111:0/1536594091 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88031640 con 0x7f6e980d1780 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.039+0000 7f6e967fc640 1 -- 172.21.15.111:0/1536594091 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880317a0 con 0x7f6e980d1780 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.039+0000 7f6e967fc640 1 -- 172.21.15.111:0/1536594091 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88031b30 con 0x7f6e980d1780 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.039+0000 7f6e9e105640 1 -- 172.21.15.111:0/1536594091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 msgr2=0x7f6e980d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.039+0000 7f6e9e105640 1 --2- 172.21.15.111:0/1536594091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980d01e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009870 tx=0x7f6e8802b8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.040+0000 7f6e9e105640 1 -- 172.21.15.111:0/1536594091 shutdown_connections 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.040+0000 7f6e9e105640 1 --2- 172.21.15.111:0/1536594091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b2ad0 0x7f6e980ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.040+0000 7f6e9e105640 1 --2- 172.21.15.111:0/1536594091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980d01e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.040+0000 7f6e9e105640 1 -- 172.21.15.111:0/1536594091 >> 172.21.15.111:0/1536594091 conn(0x7f6e9809c4e0 msgr2=0x7f6e980d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.040+0000 7f6e9e105640 1 -- 172.21.15.111:0/1536594091 shutdown_connections 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.040+0000 7f6e9e105640 1 -- 172.21.15.111:0/1536594091 wait complete. 2023-12-04T05:39:47.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.041+0000 7f6e9e105640 1 Processor -- start 2023-12-04T05:39:47.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.041+0000 7f6e9e105640 1 -- start start 2023-12-04T05:39:47.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b20a0 0x7f6e98160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e980b2ad0 0x7f6e980ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980b6d20 con 0x7f6e980d1780 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980b5dc0 con 0x7f6e980b2ad0 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980b60a0 con 0x7f6e980b20a0 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36768/0 (socket says 172.21.15.111:36768) 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 -- 172.21.15.111:0/4083183883 learned_addr learned my addr 172.21.15.111:0/4083183883 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e977fe640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b20a0 0x7f6e98160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e96ffd640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e980b2ad0 0x7f6e980ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 -- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b20a0 msgr2=0x7f6e98160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b20a0 0x7f6e98160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 -- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e980b2ad0 msgr2=0x7f6e980ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e980b2ad0 0x7f6e980ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 -- 172.21.15.111:0/4083183883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880094d0 con 0x7f6e980d1780 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e977fe640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b20a0 0x7f6e98160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e96ffd640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e980b2ad0 0x7f6e980ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e97fff640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980cb040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e90007e40 tx=0x7f6e9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90014620 con 0x7f6e980d1780 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9000ee90 con 0x7f6e980d1780 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e980b6380 con 0x7f6e980d1780 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900149e0 con 0x7f6e980d1780 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.042+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e980b6940 con 0x7f6e980d1780 2023-12-04T05:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.043+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e980b6e60 con 0x7f6e980d1780 2023-12-04T05:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.044+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6e90014b40 con 0x7f6e980d1780 2023-12-04T05:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.044+0000 7f6e7ffff640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6e8406b2a0 0x7f6e8406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.044+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e9008a540 con 0x7f6e980d1780 2023-12-04T05:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.044+0000 7f6e977fe640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6e8406b2a0 0x7f6e8406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.044+0000 7f6e977fe640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6e8406b2a0 0x7f6e8406d760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e88036000 tx=0x7f6e8803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.048+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e980b6e60 con 0x7f6e980d1780 2023-12-04T05:39:47.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.215+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e980b6e60 con 0x7f6e980d1780 2023-12-04T05:39:47.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.215+0000 7f6e7ffff640 1 -- 172.21.15.111:0/4083183883 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e980b6e60 con 0x7f6e980d1780 2023-12-04T05:39:47.217 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6e8406b2a0 msgr2=0x7f6e8406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6e8406b2a0 0x7f6e8406d760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e88036000 tx=0x7f6e8803c040 comp rx=0 tx=0).stop 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 msgr2=0x7f6e980cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980cb040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e90007e40 tx=0x7f6e9000e4b0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 shutdown_connections 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6e8406b2a0 0x7f6e8406d760 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e980d1780 0x7f6e980cb040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e980b2ad0 0x7f6e980ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 --2- 172.21.15.111:0/4083183883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e980b20a0 0x7f6e98160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.217+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 >> 172.21.15.111:0/4083183883 conn(0x7f6e9809c4e0 msgr2=0x7f6e9806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:47.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.218+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 shutdown_connections 2023-12-04T05:39:47.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.218+0000 7f6e9e105640 1 -- 172.21.15.111:0/4083183883 wait complete. 2023-12-04T05:39:47.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:47.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.334+0000 7fb18091d640 1 Processor -- start 2023-12-04T05:39:47.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.334+0000 7fb18091d640 1 -- start start 2023-12-04T05:39:47.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb18091d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c15d420 0x7fb17c0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb18091d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb17c0b5920 0x7fb17c0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb18091d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c15d800 con 0x7fb17c0d2710 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb18091d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c0d2fa0 con 0x7fb17c0b5920 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb18091d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0d2280 con 0x7fb17c15d420 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb179d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb17c0d2710 0x7fb17c0b5210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37938/0 (socket says 172.21.15.111:37938) 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb17c0b5920 0x7fb17c0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb179d9b640 1 -- 172.21.15.111:0/524917240 learned_addr learned my addr 172.21.15.111:0/524917240 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17a59c640 1 --2- 172.21.15.111:0/524917240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c15d420 0x7fb17c0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17ad9d640 1 -- 172.21.15.111:0/524917240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c15d420 msgr2=0x7fb17c0b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17ad9d640 1 --2- 172.21.15.111:0/524917240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c15d420 0x7fb17c0b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17ad9d640 1 -- 172.21.15.111:0/524917240 >> v1:172.21.15.111:6789/0 conn(0x7fb17c0d2710 legacy=0x7fb17c0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17ad9d640 1 -- 172.21.15.111:0/524917240 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0d1670 con 0x7fb17c0b5920 2023-12-04T05:39:47.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.335+0000 7fb17959a640 1 -- 172.21.15.111:0/524917240 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36736572 0 0) 0x7fb17c15d800 con 0x7fb17c0d2710 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.336+0000 7fb17ad9d640 1 --2- 172.21.15.111:0/524917240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c0b5920 0x7fb17c0b6360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb174009d90 tx=0x7fb17402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b630cbc02260ed92 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.336+0000 7fb17959a640 1 -- 172.21.15.111:0/524917240 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17402cc50 con 0x7fb17c0b5920 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.336+0000 7fb17959a640 1 -- 172.21.15.111:0/524917240 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17402cdb0 con 0x7fb17c0b5920 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.336+0000 7fb17959a640 1 -- 172.21.15.111:0/524917240 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174034660 con 0x7fb17c0b5920 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.336+0000 7fb18091d640 1 -- 172.21.15.111:0/524917240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c0b5920 msgr2=0x7fb17c0b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.336+0000 7fb18091d640 1 --2- 172.21.15.111:0/524917240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c0b5920 0x7fb17c0b6360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb174009d90 tx=0x7fb17402bea0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.337+0000 7fb18091d640 1 -- 172.21.15.111:0/524917240 shutdown_connections 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.337+0000 7fb18091d640 1 --2- 172.21.15.111:0/524917240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c0b5920 0x7fb17c0b6360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.337+0000 7fb18091d640 1 --2- 172.21.15.111:0/524917240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c15d420 0x7fb17c0b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.337+0000 7fb18091d640 1 -- 172.21.15.111:0/524917240 >> 172.21.15.111:0/524917240 conn(0x7fb17c072010 msgr2=0x7fb17c072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:47.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.337+0000 7fb18091d640 1 -- 172.21.15.111:0/524917240 shutdown_connections 2023-12-04T05:39:47.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.337+0000 7fb18091d640 1 -- 172.21.15.111:0/524917240 wait complete. 2023-12-04T05:39:47.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.338+0000 7fb18091d640 1 Processor -- start 2023-12-04T05:39:47.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.338+0000 7fb18091d640 1 -- start start 2023-12-04T05:39:47.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb17c0b5920 0x7fb17c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c0d2710 0x7fb17c1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 0x7fb17c1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1604a0 con 0x7fb17c15d420 2023-12-04T05:39:47.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1f6d50 con 0x7fb17c0b5920 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c153d10 con 0x7fb17c0d2710 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 0x7fb17c1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 0x7fb17c1537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36790/0 (socket says 172.21.15.111:36790) 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 -- 172.21.15.111:0/1026284534 learned_addr learned my addr 172.21.15.111:0/1026284534 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb179d9b640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c0d2710 0x7fb17c1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17a59c640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb17c0b5920 0x7fb17c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 -- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c0d2710 msgr2=0x7fb17c1f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c0d2710 0x7fb17c1f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 -- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb17c0b5920 msgr2=0x7fb17c1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb17c0b5920 0x7fb17c1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 -- 172.21.15.111:0/1026284534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174009a70 con 0x7fb17c15d420 2023-12-04T05:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb17ad9d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 0x7fb17c1537d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb174009eb0 tx=0x7fb174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174040650 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1740407d0 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c0c6de0 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174040b80 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.339+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c0c73a0 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.340+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c069360 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.341+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb174040ce0 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.341+0000 7fb16affd640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb15c06b2a0 0x7fb15c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.341+0000 7fb17a59c640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb15c06b2a0 0x7fb15c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.341+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1740ac960 con 0x7fb17c15d420 2023-12-04T05:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.342+0000 7fb17a59c640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb15c06b2a0 0x7fb15c06d760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb16c006fd0 tx=0x7fb16c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.346+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb174077860 con 0x7fb17c15d420 2023-12-04T05:39:47.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.512+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb17c06a4c0 con 0x7fb17c15d420 2023-12-04T05:39:47.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.512+0000 7fb16affd640 1 -- 172.21.15.111:0/1026284534 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb17407ae80 con 0x7fb17c15d420 2023-12-04T05:39:47.514 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb15c06b2a0 msgr2=0x7fb15c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb15c06b2a0 0x7fb15c06d760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb16c006fd0 tx=0x7fb16c008040 comp rx=0 tx=0).stop 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 msgr2=0x7fb17c1537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 0x7fb17c1537d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb174009eb0 tx=0x7fb174004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 shutdown_connections 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb15c06b2a0 0x7fb15c06d760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb17c15d420 0x7fb17c1537d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb17c0d2710 0x7fb17c1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 --2- 172.21.15.111:0/1026284534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb17c0b5920 0x7fb17c1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 >> 172.21.15.111:0/1026284534 conn(0x7fb17c072010 msgr2=0x7fb17c15b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 shutdown_connections 2023-12-04T05:39:47.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.514+0000 7fb18091d640 1 -- 172.21.15.111:0/1026284534 wait complete. 2023-12-04T05:39:47.527 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:47.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.626+0000 7f3ec60d3640 1 Processor -- start 2023-12-04T05:39:47.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.626+0000 7f3ec60d3640 1 -- start start 2023-12-04T05:39:47.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ec60d3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ec00b20a0 0x7f3ec00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ec60d3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec00cdcb0 con 0x7f3ec00d1780 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ec60d3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec00d39e0 con 0x7f3ec00b2ad0 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ec60d3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec00d3cc0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebeffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ec00b20a0 0x7f3ec00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebf7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3ec00d1780 0x7f3ec00d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37954/0 (socket says 172.21.15.111:37954) 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebf7fe640 1 -- 172.21.15.111:0/3869503405 learned_addr learned my addr 172.21.15.111:0/3869503405 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebeffd640 1 -- 172.21.15.111:0/3869503405 >> v1:172.21.15.192:6789/0 conn(0x7f3ec00b2ad0 legacy=0x7f3ec00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebeffd640 1 -- 172.21.15.111:0/3869503405 >> v1:172.21.15.111:6789/0 conn(0x7f3ec00d1780 legacy=0x7f3ec00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebeffd640 1 -- 172.21.15.111:0/3869503405 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec00cdf50 con 0x7f3ec00b20a0 2023-12-04T05:39:47.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.627+0000 7f3ebe7fc640 1 -- 172.21.15.111:0/3869503405 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850108084 0 0) 0x7f3ec00cdcb0 con 0x7f3ec00d1780 2023-12-04T05:39:47.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ebeffd640 1 --2- 172.21.15.111:0/3869503405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec00d01e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40097a0 tx=0x7f3eb402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b07b0c75fb0e633 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ebe7fc640 1 -- 172.21.15.111:0/3869503405 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb402e070 con 0x7f3ec00b20a0 2023-12-04T05:39:47.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ebe7fc640 1 -- 172.21.15.111:0/3869503405 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb40315a0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ebe7fc640 1 -- 172.21.15.111:0/3869503405 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4031930 con 0x7f3ec00b20a0 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3869503405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 msgr2=0x7f3ec00d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3869503405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec00d01e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40097a0 tx=0x7f3eb402b840 comp rx=0 tx=0).stop 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3869503405 shutdown_connections 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3869503405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec00d01e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3869503405 >> 172.21.15.111:0/3869503405 conn(0x7f3ec009c4e0 msgr2=0x7f3ec00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3869503405 shutdown_connections 2023-12-04T05:39:47.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.628+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3869503405 wait complete. 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 Processor -- start 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 -- start start 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec0157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ec00b2ad0 0x7f3ec00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ec00d1780 0x7f3ec00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec01f6930 con 0x7f3ec00b20a0 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec01f6c10 con 0x7f3ec00b2ad0 2023-12-04T05:39:47.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.630+0000 7f3ec60d3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec01f6ef0 con 0x7f3ec00d1780 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec0157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec0157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36820/0 (socket says 172.21.15.111:36820) 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 -- 172.21.15.111:0/3241543944 learned_addr learned my addr 172.21.15.111:0/3241543944 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebffff640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ec00d1780 0x7f3ec00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebeffd640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ec00b2ad0 0x7f3ec00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 -- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ec00d1780 msgr2=0x7f3ec00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ec00d1780 0x7f3ec00ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 -- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ec00b2ad0 msgr2=0x7f3ec00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ec00b2ad0 0x7f3ec00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 -- 172.21.15.111:0/3241543944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4009400 con 0x7f3ec00b20a0 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebffff640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ec00d1780 0x7f3ec00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebeffd640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ec00b2ad0 0x7f3ec00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ebf7fe640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec0157aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000c430 tx=0x7f3eb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00074e0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb0007660 con 0x7f3ec00b20a0 2023-12-04T05:39:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec01f71d0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0004a20 con 0x7f3ec00b20a0 2023-12-04T05:39:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.631+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec01f76f0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.632+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec00693d0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.633+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3eb0004b80 con 0x7f3ec00b20a0 2023-12-04T05:39:47.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.633+0000 7f3ea7fff640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3eac06b2a0 0x7f3eac06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:47.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.633+0000 7f3ebeffd640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3eac06b2a0 0x7f3eac06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:47.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.633+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eb00899d0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.634+0000 7f3ebeffd640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3eac06b2a0 0x7f3eac06d760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4036000 tx=0x7f3eb403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:47.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.637+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3eb00548d0 con 0x7f3ec00b20a0 2023-12-04T05:39:47.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.815+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3ec01f6a70 con 0x7f3ec00b20a0 2023-12-04T05:39:47.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.815+0000 7f3ea7fff640 1 -- 172.21.15.111:0/3241543944 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3ec01f6a70 con 0x7f3ec00b20a0 2023-12-04T05:39:47.817 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3eac06b2a0 msgr2=0x7f3eac06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3eac06b2a0 0x7f3eac06d760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4036000 tx=0x7f3eb403d040 comp rx=0 tx=0).stop 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 msgr2=0x7f3ec0157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec0157aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000c430 tx=0x7f3eb000c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 shutdown_connections 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3eac06b2a0 0x7f3eac06d760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ec00d1780 0x7f3ec00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ec00b2ad0 0x7f3ec00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 --2- 172.21.15.111:0/3241543944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ec00b20a0 0x7f3ec0157aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 >> 172.21.15.111:0/3241543944 conn(0x7f3ec009c4e0 msgr2=0x7f3ec00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:47.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 shutdown_connections 2023-12-04T05:39:47.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:47.817+0000 7f3ec60d3640 1 -- 172.21.15.111:0/3241543944 wait complete. 2023-12-04T05:39:47.829 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-12-04T05:39:47.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:47.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-04T05:39:47.856 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:39:47.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:39:47.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:47.856 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-04T05:39:47.918 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:39:47.918 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:47.938 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:39:47.938 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:47.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:48.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.094+0000 7fd4b273c640 1 Processor -- start 2023-12-04T05:39:48.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.094+0000 7fd4b273c640 1 -- start start 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4b273c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4ac144430 0x7fd4ac15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4b273c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4ac0703f0 0x7fd4ac071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4b273c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac070bd0 con 0x7fd4ac0721b0 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4b273c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac144810 con 0x7fd4ac0703f0 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4b273c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac070eb0 con 0x7fd4ac144430 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4ac0703f0 0x7fd4ac071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4ac0703f0 0x7fd4ac071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36826/0 (socket says 172.21.15.111:36826) 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 -- 172.21.15.111:0/2330915718 learned_addr learned my addr 172.21.15.111:0/2330915718 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4abfff640 1 --2- 172.21.15.111:0/2330915718 >> v2:172.21.15.192:3300/0 conn(0x7fd4ac144430 0x7fd4ac15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 -- 172.21.15.111:0/2330915718 >> v2:172.21.15.192:3300/0 conn(0x7fd4ac144430 msgr2=0x7fd4ac15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 --2- 172.21.15.111:0/2330915718 >> v2:172.21.15.192:3300/0 conn(0x7fd4ac144430 0x7fd4ac15f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 -- 172.21.15.111:0/2330915718 >> v1:172.21.15.192:6789/0 conn(0x7fd4ac0721b0 legacy=0x7fd4ac06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.095+0000 7fd4ab7fe640 1 -- 172.21.15.111:0/2330915718 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac071190 con 0x7fd4ac0703f0 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.096+0000 7fd4ab7fe640 1 --2- 172.21.15.111:0/2330915718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0703f0 0x7fd4ac071c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06e850 tx=0x7fd4a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaf5eecc6481d08 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.096+0000 7fd4aaffd640 1 -- 172.21.15.111:0/2330915718 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031870 con 0x7fd4ac0703f0 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.096+0000 7fd4aaffd640 1 -- 172.21.15.111:0/2330915718 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a00319d0 con 0x7fd4ac0703f0 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.096+0000 7fd4aaffd640 1 -- 172.21.15.111:0/2330915718 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031d60 con 0x7fd4ac0703f0 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 -- 172.21.15.111:0/2330915718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0703f0 msgr2=0x7fd4ac071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 --2- 172.21.15.111:0/2330915718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0703f0 0x7fd4ac071c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06e850 tx=0x7fd4a002b910 comp rx=0 tx=0).stop 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 -- 172.21.15.111:0/2330915718 shutdown_connections 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 --2- 172.21.15.111:0/2330915718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0703f0 0x7fd4ac071c70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 --2- 172.21.15.111:0/2330915718 >> v2:172.21.15.192:3300/0 conn(0x7fd4ac144430 0x7fd4ac15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 -- 172.21.15.111:0/2330915718 >> 172.21.15.111:0/2330915718 conn(0x7fd4ac06a4c0 msgr2=0x7fd4ac06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 -- 172.21.15.111:0/2330915718 shutdown_connections 2023-12-04T05:39:48.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.097+0000 7fd4b273c640 1 -- 172.21.15.111:0/2330915718 wait complete. 2023-12-04T05:39:48.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.098+0000 7fd4b273c640 1 Processor -- start 2023-12-04T05:39:48.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.098+0000 7fd4b273c640 1 -- start start 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b273c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0703f0 0x7fd4ac13f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b273c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac0721b0 0x7fd4ac0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b273c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 0x7fd4ac13a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b273c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac13f340 con 0x7fd4ac144430 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b273c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac13e730 con 0x7fd4ac0721b0 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b273c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac13ea10 con 0x7fd4ac0703f0 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 0x7fd4ac13a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 0x7fd4ac13a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36842/0 (socket says 172.21.15.111:36842) 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 -- 172.21.15.111:0/3334621184 learned_addr learned my addr 172.21.15.111:0/3334621184 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4ab7fe640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac0721b0 0x7fd4ac0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4abfff640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0703f0 0x7fd4ac13f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 -- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0703f0 msgr2=0x7fd4ac13f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0703f0 0x7fd4ac13f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 -- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac0721b0 msgr2=0x7fd4ac0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac0721b0 0x7fd4ac0afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 -- 172.21.15.111:0/3334621184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00094d0 con 0x7fd4ac144430 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4ab7fe640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac0721b0 0x7fd4ac0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4b0cd9640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 0x7fd4ac13a5a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400c6d0 tx=0x7fd4a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400a840 con 0x7fd4ac144430 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.099+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a400a9a0 con 0x7fd4ac144430 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.100+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac13ecf0 con 0x7fd4ac144430 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.100+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400ad90 con 0x7fd4ac144430 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.100+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac13e390 con 0x7fd4ac144430 2023-12-04T05:39:48.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.100+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac13f4b0 con 0x7fd4ac144430 2023-12-04T05:39:48.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.101+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd4a4012630 con 0x7fd4ac144430 2023-12-04T05:39:48.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.101+0000 7fd4a8ff9640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd49806b2a0 0x7fd49806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.101+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4a408a360 con 0x7fd4ac144430 2023-12-04T05:39:48.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.101+0000 7fd4abfff640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd49806b2a0 0x7fd49806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.102+0000 7fd4abfff640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd49806b2a0 0x7fd49806d760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd49c006fd0 tx=0x7fd49c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.105+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4a40551d0 con 0x7fd4ac144430 2023-12-04T05:39:48.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.274+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4ac0693d0 con 0x7fd4ac144430 2023-12-04T05:39:48.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.276+0000 7fd4a8ff9640 1 -- 172.21.15.111:0/3334621184 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fd4a40587f0 con 0x7fd4ac144430 2023-12-04T05:39:48.277 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:48.277 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.277+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd49806b2a0 msgr2=0x7fd49806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.277+0000 7fd4b273c640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd49806b2a0 0x7fd49806d760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd49c006fd0 tx=0x7fd49c008040 comp rx=0 tx=0).stop 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.277+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 msgr2=0x7fd4ac13a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.277+0000 7fd4b273c640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 0x7fd4ac13a5a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400c6d0 tx=0x7fd4a400cb90 comp rx=0 tx=0).stop 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 shutdown_connections 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd49806b2a0 0x7fd49806d760 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac144430 0x7fd4ac13a5a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac0721b0 0x7fd4ac0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 --2- 172.21.15.111:0/3334621184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0703f0 0x7fd4ac13f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 >> 172.21.15.111:0/3334621184 conn(0x7fd4ac06a4c0 msgr2=0x7fd4ac15e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 shutdown_connections 2023-12-04T05:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.278+0000 7fd4b273c640 1 -- 172.21.15.111:0/3334621184 wait complete. 2023-12-04T05:39:48.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:48.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:48.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.392+0000 7f5c7895a640 1 Processor -- start 2023-12-04T05:39:48.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.392+0000 7f5c7895a640 1 -- start start 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7895a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5c7413f640 0x7f5c7413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7895a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5c74071d20 0x7f5c74072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7895a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7413dd10 con 0x7f5c7413cee0 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7895a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c74072670 con 0x7f5c7413f640 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7895a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7413d2c0 con 0x7f5c74071d20 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5c7413cee0 0x7f5c7413d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37964/0 (socket says 172.21.15.111:37964) 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7259c640 1 -- 172.21.15.111:0/316623242 learned_addr learned my addr 172.21.15.111:0/316623242 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:48.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c71d9b640 1 --2- 172.21.15.111:0/316623242 >> v2:172.21.15.111:3300/0 conn(0x7f5c7413f640 0x7f5c7413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c72d9d640 1 --2- 172.21.15.111:0/316623242 >> v2:172.21.15.192:3300/0 conn(0x7f5c74071d20 0x7f5c74072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.393+0000 7f5c7159a640 1 -- 172.21.15.111:0/316623242 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632428330 0 0) 0x7f5c7413dd10 con 0x7f5c7413cee0 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c7159a640 1 -- 172.21.15.111:0/316623242 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c60003520 con 0x7f5c7413cee0 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c71d9b640 1 -- 172.21.15.111:0/316623242 >> v2:172.21.15.192:3300/0 conn(0x7f5c74071d20 msgr2=0x7f5c74072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c71d9b640 1 --2- 172.21.15.111:0/316623242 >> v2:172.21.15.192:3300/0 conn(0x7f5c74071d20 0x7f5c74072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c71d9b640 1 -- 172.21.15.111:0/316623242 >> v1:172.21.15.111:6789/0 conn(0x7f5c7413cee0 legacy=0x7f5c7413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c71d9b640 1 -- 172.21.15.111:0/316623242 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c741400e0 con 0x7f5c7413f640 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c71d9b640 1 --2- 172.21.15.111:0/316623242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c7413fab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c7406bdd0 tx=0x7f5c5c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e946ad0a9bab5b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c7159a640 1 -- 172.21.15.111:0/316623242 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c031710 con 0x7f5c7413f640 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c7159a640 1 -- 172.21.15.111:0/316623242 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c02f050 con 0x7f5c7413f640 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.394+0000 7f5c7159a640 1 -- 172.21.15.111:0/316623242 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c031a90 con 0x7f5c7413f640 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 -- 172.21.15.111:0/316623242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 msgr2=0x7f5c7413fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 --2- 172.21.15.111:0/316623242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c7413fab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5c7406bdd0 tx=0x7f5c5c02b7b0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 -- 172.21.15.111:0/316623242 shutdown_connections 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 --2- 172.21.15.111:0/316623242 >> v2:172.21.15.192:3300/0 conn(0x7f5c74071d20 0x7f5c74072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 --2- 172.21.15.111:0/316623242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c7413fab0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 -- 172.21.15.111:0/316623242 >> 172.21.15.111:0/316623242 conn(0x7f5c740b0cf0 msgr2=0x7f5c740b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 -- 172.21.15.111:0/316623242 shutdown_connections 2023-12-04T05:39:48.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.395+0000 7f5c7895a640 1 -- 172.21.15.111:0/316623242 wait complete. 2023-12-04T05:39:48.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.396+0000 7f5c7895a640 1 Processor -- start 2023-12-04T05:39:48.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.396+0000 7f5c7895a640 1 -- start start 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7895a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c74071d20 0x7f5c74136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7895a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7413cee0 0x7f5c7415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7895a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c741537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7895a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74136d70 con 0x7f5c7413f640 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7895a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740b31c0 con 0x7f5c74071d20 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7895a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c741524d0 con 0x7f5c7413cee0 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c741537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c741537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36860/0 (socket says 172.21.15.111:36860) 2023-12-04T05:39:48.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c72d9d640 1 -- 172.21.15.111:0/2689109196 learned_addr learned my addr 172.21.15.111:0/2689109196 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c71d9b640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7413cee0 0x7f5c7415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.397+0000 7f5c7259c640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c74071d20 0x7f5c74136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c72d9d640 1 -- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7413cee0 msgr2=0x7f5c7415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c72d9d640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7413cee0 0x7f5c7415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c72d9d640 1 -- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c74071d20 msgr2=0x7f5c74136830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c72d9d640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c74071d20 0x7f5c74136830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c72d9d640 1 -- 172.21.15.111:0/2689109196 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c009400 con 0x7f5c7413f640 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c72d9d640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c741537e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e7f0 tx=0x7f5c6c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c009510 con 0x7f5c7413f640 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c00ee40 con 0x7f5c7413f640 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c741527b0 con 0x7f5c7413f640 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00cb80 con 0x7f5c7413f640 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.398+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740b2de0 con 0x7f5c7413f640 2023-12-04T05:39:48.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.399+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c740b3330 con 0x7f5c7413f640 2023-12-04T05:39:48.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.399+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5c6c00cce0 con 0x7f5c7413f640 2023-12-04T05:39:48.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.400+0000 7f5c6affd640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5406b2d0 0x7f5c5406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.400+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c6c089b40 con 0x7f5c7413f640 2023-12-04T05:39:48.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.400+0000 7f5c7259c640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5406b2d0 0x7f5c5406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.400+0000 7f5c7259c640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5406b2d0 0x7f5c5406d790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005c00 tx=0x7f5c64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.404+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c6c054af0 con 0x7f5c7413f640 2023-12-04T05:39:48.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.569+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c74069360 con 0x7f5c7413f640 2023-12-04T05:39:48.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.569+0000 7f5c6affd640 1 -- 172.21.15.111:0/2689109196 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c6c0170a0 con 0x7f5c7413f640 2023-12-04T05:39:48.571 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:48.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5406b2d0 msgr2=0x7f5c5406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5406b2d0 0x7f5c5406d790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005c00 tx=0x7f5c64009040 comp rx=0 tx=0).stop 2023-12-04T05:39:48.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 msgr2=0x7f5c741537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c741537e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e7f0 tx=0x7f5c6c00ebf0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 shutdown_connections 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5406b2d0 0x7f5c5406d790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c7413f640 0x7f5c741537e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7413cee0 0x7f5c7415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 --2- 172.21.15.111:0/2689109196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c74071d20 0x7f5c74136830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 >> 172.21.15.111:0/2689109196 conn(0x7f5c740b0cf0 msgr2=0x7f5c740ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 shutdown_connections 2023-12-04T05:39:48.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.571+0000 7f5c7895a640 1 -- 172.21.15.111:0/2689109196 wait complete. 2023-12-04T05:39:48.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:48.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.686+0000 7f73b1f2c640 1 Processor -- start 2023-12-04T05:39:48.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.686+0000 7f73b1f2c640 1 -- start start 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73b1f2c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73ac146b10 0x7f73ac161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73b1f2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f73ac0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73b1f2c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac0d18b0 con 0x7f73ac15d8a0 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73b1f2c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac06eaf0 con 0x7f73ac146b10 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73b1f2c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15e4e0 con 0x7f73ac0d0f10 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73abfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f73ac15d8a0 0x7f73ac15dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37976/0 (socket says 172.21.15.111:37976) 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.687+0000 7f73abfff640 1 -- 172.21.15.111:0/364485384 learned_addr learned my addr 172.21.15.111:0/364485384 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f73ac0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344725310 0 0) 0x7f73ac0d18b0 con 0x7f73ac15d8a0 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73ab7fe640 1 --2- 172.21.15.111:0/364485384 >> v2:172.21.15.192:3300/0 conn(0x7f73ac146b10 0x7f73ac161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7398003520 con 0x7f73ac15d8a0 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234744246 0 0) 0x7f7398003520 con 0x7f73ac15d8a0 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 msgr2=0x7f73ac0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f73ac0d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 >> v2:172.21.15.192:3300/0 conn(0x7f73ac146b10 msgr2=0x7f73ac161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 --2- 172.21.15.111:0/364485384 >> v2:172.21.15.192:3300/0 conn(0x7f73ac146b10 0x7f73ac161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac15e7c0 con 0x7f73ac15d8a0 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f73a40019f0 con 0x7f73ac15d8a0 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 >> v1:172.21.15.111:6789/0 conn(0x7f73ac15d8a0 legacy=0x7f73ac15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7398003970 0x7f7398003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f7398004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f7398005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac06eaf0 con 0x7f73980047c0 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15e4e0 con 0x7f7398003970 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739800ba60 con 0x7f73ac0d0f10 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f7398004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73ab7fe640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7398003970 0x7f7398003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73abfff640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f7398005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 -- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 msgr2=0x7f7398005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f7398005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 -- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7398003970 msgr2=0x7f7398003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7398003970 0x7f7398003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73aaffd640 1 -- 172.21.15.111:0/364485384 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7398003520 con 0x7f73980047c0 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73abfff640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac0d0f10 0x7f7398005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.688+0000 7f73ab7fe640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7398003970 0x7f7398003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.689+0000 7f73aaffd640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f7398004c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f739400ba00 tx=0x7f73980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609ca05493534145 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.689+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940325f0 con 0x7f73980047c0 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.689+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394032750 con 0x7f73980047c0 2023-12-04T05:39:48.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.689+0000 7f73aa7fc640 1 -- 172.21.15.111:0/364485384 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394032ae0 con 0x7f73980047c0 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73b1f2c640 1 -- 172.21.15.111:0/364485384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 msgr2=0x7f7398004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/364485384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f7398004c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f739400ba00 tx=0x7f73980057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73abfff640 1 -- 172.21.15.111:0/364485384 reap_dead start 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73b1f2c640 1 -- 172.21.15.111:0/364485384 shutdown_connections 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73b1f2c640 1 -- 172.21.15.111:0/364485384 >> 172.21.15.111:0/364485384 conn(0x7f73ac070af0 msgr2=0x7f73ac0662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73b1f2c640 1 -- 172.21.15.111:0/364485384 shutdown_connections 2023-12-04T05:39:48.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.690+0000 7f73b1f2c640 1 -- 172.21.15.111:0/364485384 wait complete. 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.691+0000 7f73b1f2c640 1 Processor -- start 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.691+0000 7f73b1f2c640 1 -- start start 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f73ac15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac141400 0x7f73ac13f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73ac13fe00 0x7f73ac0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac140360 con 0x7f73980047c0 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac140640 con 0x7f73ac13fe00 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac140920 con 0x7f73ac141400 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f73ac15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f73ac15c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36870/0 (socket says 172.21.15.111:36870) 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 -- 172.21.15.111:0/980642375 learned_addr learned my addr 172.21.15.111:0/980642375 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 -- 172.21.15.111:0/980642375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac141400 msgr2=0x7f73ac13f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac141400 0x7f73ac13f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 -- 172.21.15.111:0/980642375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73ac13fe00 msgr2=0x7f73ac0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73ac13fe00 0x7f73ac0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 -- 172.21.15.111:0/980642375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac140c00 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73ab7fe640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f73ac15c130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f739c00f4e0 tx=0x7f739c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c014870 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c0149d0 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac140ee0 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c014d80 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.692+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac15c670 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.693+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac069360 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.694+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f739c01d400 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.694+0000 7f73a3fff640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2d0 0x7f738c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.694+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f739c08a370 con 0x7f73980047c0 2023-12-04T05:39:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.694+0000 7f73aaffd640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2d0 0x7f738c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.695+0000 7f73aaffd640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2d0 0x7f738c06d790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73980057a0 tx=0x7f7394032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.698+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f739c055150 con 0x7f73980047c0 2023-12-04T05:39:48.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.864+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73ac0b3950 con 0x7f73980047c0 2023-12-04T05:39:48.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.864+0000 7f73a3fff640 1 -- 172.21.15.111:0/980642375 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f739c004400 con 0x7f73980047c0 2023-12-04T05:39:48.866 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2d0 msgr2=0x7f738c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2d0 0x7f738c06d790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73980057a0 tx=0x7f7394032000 comp rx=0 tx=0).stop 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 msgr2=0x7f73ac15c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f73ac15c130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f739c00f4e0 tx=0x7f739c004610 comp rx=0 tx=0).stop 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 shutdown_connections 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f738c06b2d0 0x7f738c06d790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73ac13fe00 0x7f73ac0b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73ac141400 0x7f73ac13f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 --2- 172.21.15.111:0/980642375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73980047c0 0x7f73ac15c130 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 >> 172.21.15.111:0/980642375 conn(0x7f73ac070af0 msgr2=0x7f73ac15fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 shutdown_connections 2023-12-04T05:39:48.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.866+0000 7f73b1f2c640 1 -- 172.21.15.111:0/980642375 wait complete. 2023-12-04T05:39:48.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:48.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.980+0000 7f8f5a2c2640 1 Processor -- start 2023-12-04T05:39:48.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.980+0000 7f8f5a2c2640 1 -- start start 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f5a2c2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8f5415d480 0x7f8f540d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f5a2c2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5415ddc0 con 0x7f8f540d1310 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f5a2c2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5415e0a0 con 0x7f8f540d01e0 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f5a2c2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5415e380 con 0x7f8f5415d480 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f53fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8f5415d480 0x7f8f540d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f53fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8f5415d480 0x7f8f540d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36884/0 (socket says 172.21.15.111:36884) 2023-12-04T05:39:48.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f537fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8f540d1310 0x7f8f540cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37982/0 (socket says 172.21.15.111:37982) 2023-12-04T05:39:48.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.981+0000 7f8f53fff640 1 -- 172.21.15.111:0/2558914449 learned_addr learned my addr 172.21.15.111:0/2558914449 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:48.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.982+0000 7f8f53fff640 1 -- 172.21.15.111:0/2558914449 >> v1:172.21.15.192:6789/0 conn(0x7f8f540d01e0 legacy=0x7f8f54160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:48.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.982+0000 7f8f53fff640 1 -- 172.21.15.111:0/2558914449 >> v1:172.21.15.111:6789/0 conn(0x7f8f540d1310 legacy=0x7f8f540cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.982+0000 7f8f53fff640 1 -- 172.21.15.111:0/2558914449 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5415e660 con 0x7f8f5415d480 2023-12-04T05:39:48.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.982+0000 7f8f52ffd640 1 -- 172.21.15.111:0/2558914449 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260586849 0 0) 0x7f8f5415ddc0 con 0x7f8f540d1310 2023-12-04T05:39:48.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.982+0000 7f8f53fff640 1 --2- 172.21.15.111:0/2558914449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f5415d480 0x7f8f540d0dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009870 tx=0x7f8f4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e58ccd97692825 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f52ffd640 1 -- 172.21.15.111:0/2558914449 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44030860 con 0x7f8f5415d480 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f52ffd640 1 -- 172.21.15.111:0/2558914449 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f440309c0 con 0x7f8f5415d480 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f52ffd640 1 -- 172.21.15.111:0/2558914449 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44030d50 con 0x7f8f5415d480 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/2558914449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f5415d480 msgr2=0x7f8f540d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/2558914449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f5415d480 0x7f8f540d0dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009870 tx=0x7f8f4402b9a0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/2558914449 shutdown_connections 2023-12-04T05:39:48.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/2558914449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f5415d480 0x7f8f540d0dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/2558914449 >> 172.21.15.111:0/2558914449 conn(0x7f8f5409c540 msgr2=0x7f8f540d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/2558914449 shutdown_connections 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.983+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/2558914449 wait complete. 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.984+0000 7f8f5a2c2640 1 Processor -- start 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.984+0000 7f8f5a2c2640 1 -- start start 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5a2c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 0x7f8f540b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5a2c2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f540d1310 0x7f8f540cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5a2c2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f5415d480 0x7f8f54146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5a2c2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540cc570 con 0x7f8f540d01e0 2023-12-04T05:39:48.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5a2c2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540b90b0 con 0x7f8f5415d480 2023-12-04T05:39:48.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5a2c2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540cb960 con 0x7f8f540d1310 2023-12-04T05:39:48.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f53fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 0x7f8f540b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f53fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 0x7f8f540b8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36886/0 (socket says 172.21.15.111:36886) 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f53fff640 1 -- 172.21.15.111:0/1323893104 learned_addr learned my addr 172.21.15.111:0/1323893104 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f5885f640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f5415d480 0x7f8f54146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.985+0000 7f8f537fe640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f540d1310 0x7f8f540cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f53fff640 1 -- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f540d1310 msgr2=0x7f8f540cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f53fff640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f540d1310 0x7f8f540cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f53fff640 1 -- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f5415d480 msgr2=0x7f8f54146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f53fff640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f5415d480 0x7f8f54146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:48.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f53fff640 1 -- 172.21.15.111:0/1323893104 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440094d0 con 0x7f8f540d01e0 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f5885f640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f5415d480 0x7f8f54146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f53fff640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 0x7f8f540b8b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009870 tx=0x7f8f44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44039b60 con 0x7f8f540d01e0 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44039cc0 con 0x7f8f540d01e0 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f540cbc40 con 0x7f8f540d01e0 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44043630 con 0x7f8f540d01e0 2023-12-04T05:39:48.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.986+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f540cc200 con 0x7f8f540d01e0 2023-12-04T05:39:48.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.987+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f54067980 con 0x7f8f540d01e0 2023-12-04T05:39:48.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.987+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8f4404c3b0 con 0x7f8f540d01e0 2023-12-04T05:39:48.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.988+0000 7f8f50ff9640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8f4006b2a0 0x7f8f4006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:48.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.988+0000 7f8f537fe640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8f4006b2a0 0x7f8f4006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:48.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.988+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f440ac130 con 0x7f8f540d01e0 2023-12-04T05:39:48.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.988+0000 7f8f537fe640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8f4006b2a0 0x7f8f4006d760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f541f62b0 tx=0x7f8f48007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:48.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:48.992+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f44076f10 con 0x7f8f540d01e0 2023-12-04T05:39:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.156+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f540cc390 con 0x7f8f540d01e0 2023-12-04T05:39:49.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.157+0000 7f8f50ff9640 1 -- 172.21.15.111:0/1323893104 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f4407a530 con 0x7f8f540d01e0 2023-12-04T05:39:49.159 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:49.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8f4006b2a0 msgr2=0x7f8f4006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8f4006b2a0 0x7f8f4006d760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f541f62b0 tx=0x7f8f48007b50 comp rx=0 tx=0).stop 2023-12-04T05:39:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 msgr2=0x7f8f540b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 0x7f8f540b8b70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009870 tx=0x7f8f44004730 comp rx=0 tx=0).stop 2023-12-04T05:39:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 shutdown_connections 2023-12-04T05:39:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8f4006b2a0 0x7f8f4006d760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f5415d480 0x7f8f54146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f540d1310 0x7f8f540cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 --2- 172.21.15.111:0/1323893104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f540d01e0 0x7f8f540b8b70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.158+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 >> 172.21.15.111:0/1323893104 conn(0x7f8f5409c540 msgr2=0x7f8f5406be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.159+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 shutdown_connections 2023-12-04T05:39:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.159+0000 7f8f5a2c2640 1 -- 172.21.15.111:0/1323893104 wait complete. 2023-12-04T05:39:49.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:49.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.274+0000 7f703d82d640 1 Processor -- start 2023-12-04T05:39:49.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.274+0000 7f703d82d640 1 -- start start 2023-12-04T05:39:49.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f703d82d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f70380d1780 0x7f70380d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f703d82d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70380b2ad0 0x7f70380b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f703d82d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380cdcb0 con 0x7f70380b20a0 2023-12-04T05:39:49.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f703d82d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70380d39e0 con 0x7f70380b2ad0 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f703d82d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70380d3cc0 con 0x7f70380d1780 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f70377fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70380b2ad0 0x7f70380b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f702ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f70380b20a0 0x7f70380d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37996/0 (socket says 172.21.15.111:37996) 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f702ffff640 1 -- 172.21.15.111:0/4079696113 learned_addr learned my addr 172.21.15.111:0/4079696113 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f7036ffd640 1 --2- 172.21.15.111:0/4079696113 >> v2:172.21.15.192:3300/0 conn(0x7f70380d1780 0x7f70380d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f70377fe640 1 -- 172.21.15.111:0/4079696113 >> v2:172.21.15.192:3300/0 conn(0x7f70380d1780 msgr2=0x7f70380d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:49.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f70377fe640 1 --2- 172.21.15.111:0/4079696113 >> v2:172.21.15.192:3300/0 conn(0x7f70380d1780 0x7f70380d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f70377fe640 1 -- 172.21.15.111:0/4079696113 >> v1:172.21.15.111:6789/0 conn(0x7f70380b20a0 legacy=0x7f70380d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f70367fc640 1 -- 172.21.15.111:0/4079696113 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524043017 0 0) 0x7f70380cdcb0 con 0x7f70380b20a0 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.275+0000 7f70377fe640 1 -- 172.21.15.111:0/4079696113 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380cdf50 con 0x7f70380b2ad0 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.276+0000 7f70377fe640 1 --2- 172.21.15.111:0/4079696113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380b2ad0 0x7f70380b2f00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f703808d370 tx=0x7f703002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287e737a69a5bdbf server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.276+0000 7f70367fc640 1 -- 172.21.15.111:0/4079696113 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703002cbf0 con 0x7f70380b2ad0 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.276+0000 7f70367fc640 1 -- 172.21.15.111:0/4079696113 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703002cd50 con 0x7f70380b2ad0 2023-12-04T05:39:49.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.276+0000 7f70367fc640 1 -- 172.21.15.111:0/4079696113 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030034660 con 0x7f70380b2ad0 2023-12-04T05:39:49.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 -- 172.21.15.111:0/4079696113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380b2ad0 msgr2=0x7f70380b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 --2- 172.21.15.111:0/4079696113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380b2ad0 0x7f70380b2f00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f703808d370 tx=0x7f703002be40 comp rx=0 tx=0).stop 2023-12-04T05:39:49.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 -- 172.21.15.111:0/4079696113 shutdown_connections 2023-12-04T05:39:49.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 --2- 172.21.15.111:0/4079696113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380b2ad0 0x7f70380b2f00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 --2- 172.21.15.111:0/4079696113 >> v2:172.21.15.192:3300/0 conn(0x7f70380d1780 0x7f70380d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 -- 172.21.15.111:0/4079696113 >> 172.21.15.111:0/4079696113 conn(0x7f703809c4e0 msgr2=0x7f70380d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 -- 172.21.15.111:0/4079696113 shutdown_connections 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.277+0000 7f703d82d640 1 -- 172.21.15.111:0/4079696113 wait complete. 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.278+0000 7f703d82d640 1 Processor -- start 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.278+0000 7f703d82d640 1 -- start start 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.278+0000 7f703d82d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70380b20a0 0x7f7038157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.278+0000 7f703d82d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70380b2ad0 0x7f70380b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.278+0000 7f703d82d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 0x7f70381601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f703d82d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70380cb1d0 con 0x7f70380d1780 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f703d82d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703815e170 con 0x7f70380b20a0 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f703d82d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703815e3c0 con 0x7f70380b2ad0 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 0x7f70381601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 0x7f70381601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36906/0 (socket says 172.21.15.111:36906) 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 -- 172.21.15.111:0/3408292460 learned_addr learned my addr 172.21.15.111:0/3408292460 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f7036ffd640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70380b20a0 0x7f7038157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f702ffff640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70380b2ad0 0x7f70380b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 -- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70380b2ad0 msgr2=0x7f70380b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70380b2ad0 0x7f70380b89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 -- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70380b20a0 msgr2=0x7f7038157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70380b20a0 0x7f7038157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 -- 172.21.15.111:0/3408292460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7030009a70 con 0x7f70380d1780 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f7036ffd640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70380b20a0 0x7f7038157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f70377fe640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 0x7f70381601b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7030009e80 tx=0x7f7030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030040650 con 0x7f70380d1780 2023-12-04T05:39:49.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70300407b0 con 0x7f70380d1780 2023-12-04T05:39:49.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030040b60 con 0x7f70380d1780 2023-12-04T05:39:49.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703815e6a0 con 0x7f70380d1780 2023-12-04T05:39:49.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.279+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7038157d20 con 0x7f70380d1780 2023-12-04T05:39:49.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.280+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70380cb310 con 0x7f70380d1780 2023-12-04T05:39:49.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.281+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f7030040cc0 con 0x7f70380d1780 2023-12-04T05:39:49.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.281+0000 7f7034ff9640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f701806b2a0 0x7f701806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.281+0000 7f7036ffd640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f701806b2a0 0x7f701806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.281+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70300acd20 con 0x7f70380d1780 2023-12-04T05:39:49.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.282+0000 7f7036ffd640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f701806b2a0 0x7f701806d760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7028009af0 tx=0x7f7028008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.286+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70380cb310 con 0x7f70380d1780 2023-12-04T05:39:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.468+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f70380cb310 con 0x7f70380d1780 2023-12-04T05:39:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.468+0000 7f7034ff9640 1 -- 172.21.15.111:0/3408292460 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f70380cb310 con 0x7f70380d1780 2023-12-04T05:39:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f701806b2a0 msgr2=0x7f701806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f701806b2a0 0x7f701806d760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7028009af0 tx=0x7f7028008040 comp rx=0 tx=0).stop 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 msgr2=0x7f70381601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 0x7f70381601b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7030009e80 tx=0x7f7030004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 shutdown_connections 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f701806b2a0 0x7f701806d760 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70380d1780 0x7f70381601b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70380b2ad0 0x7f70380b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 --2- 172.21.15.111:0/3408292460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70380b20a0 0x7f7038157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 >> 172.21.15.111:0/3408292460 conn(0x7f703809c4e0 msgr2=0x7f703806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 shutdown_connections 2023-12-04T05:39:49.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.470+0000 7f703d82d640 1 -- 172.21.15.111:0/3408292460 wait complete. 2023-12-04T05:39:49.483 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:39:49.484 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:49.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:49.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.603+0000 7f59246b4640 1 Processor -- start 2023-12-04T05:39:49.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.603+0000 7f59246b4640 1 -- start start 2023-12-04T05:39:49.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f59246b4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f591c1401c0 0x7f591c071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f59246b4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f591c071710 0x7f591c06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f59246b4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c1405a0 con 0x7f591c06c790 2023-12-04T05:39:49.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f59246b4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c06cba0 con 0x7f591c071710 2023-12-04T05:39:49.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f59246b4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c140880 con 0x7f591c1401c0 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5921c4f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f591c071710 0x7f591c06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5922c51640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f591c06c790 0x7f591c0703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38010/0 (socket says 172.21.15.111:38010) 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5922c51640 1 -- 172.21.15.111:0/3178668669 learned_addr learned my addr 172.21.15.111:0/3178668669 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5922450640 1 --2- 172.21.15.111:0/3178668669 >> v2:172.21.15.192:3300/0 conn(0x7f591c1401c0 0x7f591c071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5921c4f640 1 -- 172.21.15.111:0/3178668669 >> v2:172.21.15.192:3300/0 conn(0x7f591c1401c0 msgr2=0x7f591c071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5921c4f640 1 --2- 172.21.15.111:0/3178668669 >> v2:172.21.15.192:3300/0 conn(0x7f591c1401c0 0x7f591c071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5921c4f640 1 -- 172.21.15.111:0/3178668669 >> v1:172.21.15.111:6789/0 conn(0x7f591c06c790 legacy=0x7f591c0703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f5921c4f640 1 -- 172.21.15.111:0/3178668669 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c15db00 con 0x7f591c071710 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.604+0000 7f592144e640 1 -- 172.21.15.111:0/3178668669 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257292478 0 0) 0x7f591c1405a0 con 0x7f591c06c790 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.605+0000 7f5921c4f640 1 --2- 172.21.15.111:0/3178668669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c071710 0x7f591c06c250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f591c06bdd0 tx=0x7f590c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9ed62cb985bca0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.605+0000 7f592144e640 1 -- 172.21.15.111:0/3178668669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c031830 con 0x7f591c071710 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.605+0000 7f592144e640 1 -- 172.21.15.111:0/3178668669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c031990 con 0x7f591c071710 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.605+0000 7f592144e640 1 -- 172.21.15.111:0/3178668669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c031d20 con 0x7f591c071710 2023-12-04T05:39:49.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.605+0000 7f59246b4640 1 -- 172.21.15.111:0/3178668669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c071710 msgr2=0x7f591c06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.605+0000 7f59246b4640 1 --2- 172.21.15.111:0/3178668669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c071710 0x7f591c06c250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f591c06bdd0 tx=0x7f590c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.606+0000 7f59246b4640 1 -- 172.21.15.111:0/3178668669 shutdown_connections 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.606+0000 7f59246b4640 1 --2- 172.21.15.111:0/3178668669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c071710 0x7f591c06c250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.606+0000 7f59246b4640 1 --2- 172.21.15.111:0/3178668669 >> v2:172.21.15.192:3300/0 conn(0x7f591c1401c0 0x7f591c071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.606+0000 7f59246b4640 1 -- 172.21.15.111:0/3178668669 >> 172.21.15.111:0/3178668669 conn(0x7f591c06a4c0 msgr2=0x7f591c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.606+0000 7f59246b4640 1 -- 172.21.15.111:0/3178668669 shutdown_connections 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.606+0000 7f59246b4640 1 -- 172.21.15.111:0/3178668669 wait complete. 2023-12-04T05:39:49.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 Processor -- start 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 -- start start 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f591c06c790 0x7f591c12fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f591c071710 0x7f591c0acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 0x7f591c0ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c12e530 con 0x7f591c1401c0 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c131020 con 0x7f591c06c790 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.607+0000 7f59246b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1f3b50 con 0x7f591c071710 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 0x7f591c0ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 0x7f591c0ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36920/0 (socket says 172.21.15.111:36920) 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 -- 172.21.15.111:0/3225016157 learned_addr learned my addr 172.21.15.111:0/3225016157 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922450640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f591c06c790 0x7f591c12fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5921c4f640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f591c071710 0x7f591c0acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 -- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f591c071710 msgr2=0x7f591c0acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f591c071710 0x7f591c0acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 -- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f591c06c790 msgr2=0x7f591c12fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f591c06c790 0x7f591c12fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 -- 172.21.15.111:0/3225016157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009400 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922450640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f591c06c790 0x7f591c12fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5922c51640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 0x7f591c0ad190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f591400c780 tx=0x7f591400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400ce90 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591400a640 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c1f7060 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400a9d0 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.608+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c1306f0 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.609+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c1311d0 con 0x7f591c1401c0 2023-12-04T05:39:49.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.610+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f591400ab30 con 0x7f591c1401c0 2023-12-04T05:39:49.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.610+0000 7f5912ffd640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f590006b2a0 0x7f590006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.610+0000 7f5922450640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f590006b2a0 0x7f590006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.610+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5914089050 con 0x7f591c1401c0 2023-12-04T05:39:49.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.611+0000 7f5922450640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f590006b2a0 0x7f590006d760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f591c0b0640 tx=0x7f5918008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.614+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f591c1311d0 con 0x7f591c1401c0 2023-12-04T05:39:49.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.783+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f591c1311d0 con 0x7f591c1401c0 2023-12-04T05:39:49.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.784+0000 7f5912ffd640 1 -- 172.21.15.111:0/3225016157 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f591c1311d0 con 0x7f591c1401c0 2023-12-04T05:39:49.786 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:49.786 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f590006b2a0 msgr2=0x7f590006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f590006b2a0 0x7f590006d760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f591c0b0640 tx=0x7f5918008040 comp rx=0 tx=0).stop 2023-12-04T05:39:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 msgr2=0x7f591c0ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 0x7f591c0ad190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f591400c780 tx=0x7f591400cc40 comp rx=0 tx=0).stop 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 shutdown_connections 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f590006b2a0 0x7f590006d760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f591c1401c0 0x7f591c0ad190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f591c071710 0x7f591c0acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 --2- 172.21.15.111:0/3225016157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f591c06c790 0x7f591c12fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.786+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 >> 172.21.15.111:0/3225016157 conn(0x7f591c06a4c0 msgr2=0x7f591c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.787+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 shutdown_connections 2023-12-04T05:39:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.787+0000 7f59246b4640 1 -- 172.21.15.111:0/3225016157 wait complete. 2023-12-04T05:39:49.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:49.800 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:49.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.902+0000 7fbf1c47c640 1 Processor -- start 2023-12-04T05:39:49.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.902+0000 7fbf1c47c640 1 -- start start 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1c47c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf140af750 0x7fbf140ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1c47c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf140ac100 0x7fbf140cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1c47c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1415df30 con 0x7fbf1415da80 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1c47c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf14157f60 con 0x7fbf140af750 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1c47c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140cd190 con 0x7fbf140ac100 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf140af750 0x7fbf140ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf140af750 0x7fbf140ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36930/0 (socket says 172.21.15.111:36930) 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 -- 172.21.15.111:0/4230628654 learned_addr learned my addr 172.21.15.111:0/4230628654 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1aa19640 1 --2- 172.21.15.111:0/4230628654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf140ac100 0x7fbf140cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 -- 172.21.15.111:0/4230628654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf140ac100 msgr2=0x7fbf140cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:49.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 --2- 172.21.15.111:0/4230628654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf140ac100 0x7fbf140cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 -- 172.21.15.111:0/4230628654 >> v1:172.21.15.192:6789/0 conn(0x7fbf1415da80 legacy=0x7fbf140afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:49.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.903+0000 7fbf1a218640 1 -- 172.21.15.111:0/4230628654 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1415e190 con 0x7fbf140af750 2023-12-04T05:39:49.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.904+0000 7fbf1a218640 1 --2- 172.21.15.111:0/4230628654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140ae600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf1406bdc0 tx=0x7fbf1002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122f401eef78ba44 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.904+0000 7fbf19216640 1 -- 172.21.15.111:0/4230628654 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1002e070 con 0x7fbf140af750 2023-12-04T05:39:49.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.904+0000 7fbf19216640 1 -- 172.21.15.111:0/4230628654 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf100315a0 con 0x7fbf140af750 2023-12-04T05:39:49.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.904+0000 7fbf19216640 1 -- 172.21.15.111:0/4230628654 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10031930 con 0x7fbf140af750 2023-12-04T05:39:49.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.904+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4230628654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 msgr2=0x7fbf140ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.904+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4230628654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140ae600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf1406bdc0 tx=0x7fbf1002b840 comp rx=0 tx=0).stop 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.905+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4230628654 shutdown_connections 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.905+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4230628654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf140ac100 0x7fbf140cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.905+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4230628654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140ae600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.905+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4230628654 >> 172.21.15.111:0/4230628654 conn(0x7fbf140b4640 msgr2=0x7fbf140b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.905+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4230628654 shutdown_connections 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.905+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4230628654 wait complete. 2023-12-04T05:39:49.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.906+0000 7fbf1c47c640 1 Processor -- start 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.906+0000 7fbf1c47c640 1 -- start start 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.906+0000 7fbf1c47c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf140ac100 0x7fbf14155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.906+0000 7fbf1c47c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.906+0000 7fbf1c47c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf1415da80 0x7fbf140c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.906+0000 7fbf1c47c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14155ea0 con 0x7fbf140af750 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf1c47c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140ca210 con 0x7fbf140ac100 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf1c47c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140c8c30 con 0x7fbf1415da80 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36932/0 (socket says 172.21.15.111:36932) 2023-12-04T05:39:49.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 -- 172.21.15.111:0/4088749807 learned_addr learned my addr 172.21.15.111:0/4088749807 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf1aa19640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf1415da80 0x7fbf140c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf1a218640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf140ac100 0x7fbf14155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 -- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf1415da80 msgr2=0x7fbf140c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf1415da80 0x7fbf140c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 -- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf140ac100 msgr2=0x7fbf14155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf140ac100 0x7fbf14155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 -- 172.21.15.111:0/4088749807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100094d0 con 0x7fbf140af750 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf19a17640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140c8530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b3f0 tx=0x7fbf0400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011830 con 0x7fbf140af750 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04011990 con 0x7fbf140af750 2023-12-04T05:39:49.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf140c8f10 con 0x7fbf140af750 2023-12-04T05:39:49.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011d40 con 0x7fbf140af750 2023-12-04T05:39:49.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.907+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf140c94d0 con 0x7fbf140af750 2023-12-04T05:39:49.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.908+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf14069360 con 0x7fbf140af750 2023-12-04T05:39:49.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.909+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbf0401a400 con 0x7fbf140af750 2023-12-04T05:39:49.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.909+0000 7fbf0affd640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbef806b2a0 0x7fbef806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:49.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.909+0000 7fbf1a218640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbef806b2a0 0x7fbef806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:49.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.909+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf04089bf0 con 0x7fbf140af750 2023-12-04T05:39:49.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.910+0000 7fbf1a218640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbef806b2a0 0x7fbef806d760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf10036000 tx=0x7fbf1003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:49.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:49.914+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf04054b10 con 0x7fbf140af750 2023-12-04T05:39:50.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.079+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf140b20f0 con 0x7fbf140af750 2023-12-04T05:39:50.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.079+0000 7fbf0affd640 1 -- 172.21.15.111:0/4088749807 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf04015020 con 0x7fbf140af750 2023-12-04T05:39:50.081 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbef806b2a0 msgr2=0x7fbef806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbef806b2a0 0x7fbef806d760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf10036000 tx=0x7fbf1003d040 comp rx=0 tx=0).stop 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 msgr2=0x7fbf140c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140c8530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b3f0 tx=0x7fbf0400b8b0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 shutdown_connections 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbef806b2a0 0x7fbef806d760 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf1415da80 0x7fbf140c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf140af750 0x7fbf140c8530 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 --2- 172.21.15.111:0/4088749807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf140ac100 0x7fbf14155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.081+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 >> 172.21.15.111:0/4088749807 conn(0x7fbf140b4640 msgr2=0x7fbf140cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.082+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 shutdown_connections 2023-12-04T05:39:50.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.082+0000 7fbf1c47c640 1 -- 172.21.15.111:0/4088749807 wait complete. 2023-12-04T05:39:50.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:50.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.195+0000 7fcb0593c640 1 Processor -- start 2023-12-04T05:39:50.197 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.195+0000 7fcb0593c640 1 -- start start 2023-12-04T05:39:50.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcb0593c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb00140300 0x7fcb000b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcb0593c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb000b3470 0x7fcb0006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcb0593c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0015d7f0 con 0x7fcb000b3030 2023-12-04T05:39:50.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcb0593c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb000b4e20 con 0x7fcb000b3470 2023-12-04T05:39:50.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcb0593c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001406e0 con 0x7fcb00140300 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb000b3470 0x7fcb0006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcb000b3470 0x7fcb0006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36940/0 (socket says 172.21.15.111:36940) 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 -- 172.21.15.111:0/3334153834 learned_addr learned my addr 172.21.15.111:0/3334153834 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcafeffd640 1 --2- 172.21.15.111:0/3334153834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb00140300 0x7fcb000b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 -- 172.21.15.111:0/3334153834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb00140300 msgr2=0x7fcb000b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 --2- 172.21.15.111:0/3334153834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb00140300 0x7fcb000b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 -- 172.21.15.111:0/3334153834 >> v1:172.21.15.192:6789/0 conn(0x7fcb000b3030 legacy=0x7fcb0006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.196+0000 7fcaff7fe640 1 -- 172.21.15.111:0/3334153834 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0015dac0 con 0x7fcb000b3470 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcaff7fe640 1 --2- 172.21.15.111:0/3334153834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb000b3470 0x7fcb0006eef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009870 tx=0x7fcaf802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a9e90fc362ee8f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcafdffb640 1 -- 172.21.15.111:0/3334153834 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80317c0 con 0x7fcb000b3470 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcafdffb640 1 -- 172.21.15.111:0/3334153834 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8031920 con 0x7fcb000b3470 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcafdffb640 1 -- 172.21.15.111:0/3334153834 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8031cb0 con 0x7fcb000b3470 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcb0593c640 1 -- 172.21.15.111:0/3334153834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb000b3470 msgr2=0x7fcb0006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcb0593c640 1 --2- 172.21.15.111:0/3334153834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb000b3470 0x7fcb0006eef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009870 tx=0x7fcaf802b860 comp rx=0 tx=0).stop 2023-12-04T05:39:50.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcb0593c640 1 -- 172.21.15.111:0/3334153834 shutdown_connections 2023-12-04T05:39:50.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcb0593c640 1 --2- 172.21.15.111:0/3334153834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb000b3470 0x7fcb0006eef0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.197+0000 7fcb0593c640 1 --2- 172.21.15.111:0/3334153834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb00140300 0x7fcb000b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.198+0000 7fcb0593c640 1 -- 172.21.15.111:0/3334153834 >> 172.21.15.111:0/3334153834 conn(0x7fcb0006a8b0 msgr2=0x7fcb0006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.198+0000 7fcb0593c640 1 -- 172.21.15.111:0/3334153834 shutdown_connections 2023-12-04T05:39:50.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.198+0000 7fcb0593c640 1 -- 172.21.15.111:0/3334153834 wait complete. 2023-12-04T05:39:50.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.199+0000 7fcb0593c640 1 Processor -- start 2023-12-04T05:39:50.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.199+0000 7fcb0593c640 1 -- start start 2023-12-04T05:39:50.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.199+0000 7fcb0593c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb000b3030 0x7fcb001529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb000b3470 0x7fcb00155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 0x7fcb0013c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0013d4b0 con 0x7fcb00140300 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001573e0 con 0x7fcb000b3470 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00155410 con 0x7fcb000b3030 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 0x7fcb0013c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 0x7fcb0013c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36952/0 (socket says 172.21.15.111:36952) 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 -- 172.21.15.111:0/1499240100 learned_addr learned my addr 172.21.15.111:0/1499240100 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcafeffd640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb000b3030 0x7fcb001529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcafe7fc640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb000b3470 0x7fcb00155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 -- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb000b3030 msgr2=0x7fcb001529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb000b3030 0x7fcb001529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 -- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb000b3470 msgr2=0x7fcb00155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb000b3470 0x7fcb00155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 -- 172.21.15.111:0/1499240100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80094d0 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcaff7fe640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 0x7fcb0013c650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80090b0 tx=0x7fcaf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8032920 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8032ac0 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb001556f0 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8032e70 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.200+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb001576c0 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.201+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00155c10 con 0x7fcb00140300 2023-12-04T05:39:50.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.202+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fcaf80453f0 con 0x7fcb00140300 2023-12-04T05:39:50.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.202+0000 7fcae7fff640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcaec06b2d0 0x7fcaec06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.202+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaf80ac170 con 0x7fcb00140300 2023-12-04T05:39:50.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.202+0000 7fcafeffd640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcaec06b2d0 0x7fcaec06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.203+0000 7fcafeffd640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcaec06b2d0 0x7fcaec06d790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00070b0 tx=0x7fcaf0007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.206+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb00155c10 con 0x7fcb00140300 2023-12-04T05:39:50.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.372+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb00155c10 con 0x7fcb00140300 2023-12-04T05:39:50.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.373+0000 7fcae7fff640 1 -- 172.21.15.111:0/1499240100 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb00155c10 con 0x7fcb00140300 2023-12-04T05:39:50.374 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:50.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.374+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcaec06b2d0 msgr2=0x7fcaec06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.374+0000 7fcb0593c640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcaec06b2d0 0x7fcaec06d790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00070b0 tx=0x7fcaf0007040 comp rx=0 tx=0).stop 2023-12-04T05:39:50.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.374+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 msgr2=0x7fcb0013c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.374+0000 7fcb0593c640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 0x7fcb0013c650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80090b0 tx=0x7fcaf80046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 shutdown_connections 2023-12-04T05:39:50.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcaec06b2d0 0x7fcaec06d790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb00140300 0x7fcb0013c650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb000b3470 0x7fcb00155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 --2- 172.21.15.111:0/1499240100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb000b3030 0x7fcb001529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 >> 172.21.15.111:0/1499240100 conn(0x7fcb0006a8b0 msgr2=0x7fcb00142520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 shutdown_connections 2023-12-04T05:39:50.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.375+0000 7fcb0593c640 1 -- 172.21.15.111:0/1499240100 wait complete. 2023-12-04T05:39:50.387 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:50.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.488+0000 7f999fc31640 1 Processor -- start 2023-12-04T05:39:50.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.488+0000 7f999fc31640 1 -- start start 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999fc31640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f999815e770 0x7f99980cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999fc31640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99980cc120 0x7f999815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999fc31640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9998070190 con 0x7f999809a3a0 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999fc31640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99980703e0 con 0x7f99980cc120 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999fc31640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9998099ba0 con 0x7f999815e770 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999e1ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f99980cc120 0x7f999815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999d1cc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f999809a3a0 0x7f99980cba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54786/0 (socket says 172.21.15.111:54786) 2023-12-04T05:39:50.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999d1cc640 1 -- 172.21.15.111:0/533681350 learned_addr learned my addr 172.21.15.111:0/533681350 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:50.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.489+0000 7f999d9cd640 1 --2- 172.21.15.111:0/533681350 >> v2:172.21.15.192:3300/0 conn(0x7f999815e770 0x7f99980cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999e1ce640 1 -- 172.21.15.111:0/533681350 >> v2:172.21.15.192:3300/0 conn(0x7f999815e770 msgr2=0x7f99980cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:50.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999e1ce640 1 --2- 172.21.15.111:0/533681350 >> v2:172.21.15.192:3300/0 conn(0x7f999815e770 0x7f99980cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999e1ce640 1 -- 172.21.15.111:0/533681350 >> v1:172.21.15.111:6789/0 conn(0x7f999809a3a0 legacy=0x7f99980cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999e1ce640 1 -- 172.21.15.111:0/533681350 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998099e80 con 0x7f99980cc120 2023-12-04T05:39:50.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999c9cb640 1 -- 172.21.15.111:0/533681350 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691328932 0 0) 0x7f9998070190 con 0x7f999809a3a0 2023-12-04T05:39:50.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999e1ce640 1 --2- 172.21.15.111:0/533681350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99980cc120 0x7f999815ee50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9998072b10 tx=0x7f999002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8013ae5f1821621 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.490+0000 7f999c9cb640 1 -- 172.21.15.111:0/533681350 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999002e070 con 0x7f99980cc120 2023-12-04T05:39:50.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999c9cb640 1 -- 172.21.15.111:0/533681350 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99900315a0 con 0x7f99980cc120 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999c9cb640 1 -- 172.21.15.111:0/533681350 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990031930 con 0x7f99980cc120 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 -- 172.21.15.111:0/533681350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99980cc120 msgr2=0x7f999815ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 --2- 172.21.15.111:0/533681350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99980cc120 0x7f999815ee50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9998072b10 tx=0x7f999002b840 comp rx=0 tx=0).stop 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 -- 172.21.15.111:0/533681350 shutdown_connections 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 --2- 172.21.15.111:0/533681350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99980cc120 0x7f999815ee50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 --2- 172.21.15.111:0/533681350 >> v2:172.21.15.192:3300/0 conn(0x7f999815e770 0x7f99980cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 -- 172.21.15.111:0/533681350 >> 172.21.15.111:0/533681350 conn(0x7f99980b2ad0 msgr2=0x7f99980b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 -- 172.21.15.111:0/533681350 shutdown_connections 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.491+0000 7f999fc31640 1 -- 172.21.15.111:0/533681350 wait complete. 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.492+0000 7f999fc31640 1 Processor -- start 2023-12-04T05:39:50.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.492+0000 7f999fc31640 1 -- start start 2023-12-04T05:39:50.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999fc31640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 0x7f99981f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999fc31640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99980cc120 0x7f99980c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999fc31640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999815e770 0x7f99980b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999fc31640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998158dc0 con 0x7f999809a3a0 2023-12-04T05:39:50.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999fc31640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99981581b0 con 0x7f99980cc120 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999fc31640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9998158490 con 0x7f999815e770 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 0x7f99981f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 0x7f99981f0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36132/0 (socket says 172.21.15.111:36132) 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 -- 172.21.15.111:0/1847216210 learned_addr learned my addr 172.21.15.111:0/1847216210 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 -- 172.21.15.111:0/1847216210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999815e770 msgr2=0x7f99980b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999815e770 0x7f99980b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 -- 172.21.15.111:0/1847216210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99980cc120 msgr2=0x7f99980c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99980cc120 0x7f99980c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 -- 172.21.15.111:0/1847216210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990009400 con 0x7f999809a3a0 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.493+0000 7f999d9cd640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 0x7f99981f0210 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f999400efc0 tx=0x7f999400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.494+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994014480 con 0x7f999809a3a0 2023-12-04T05:39:50.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.494+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999400f040 con 0x7f999809a3a0 2023-12-04T05:39:50.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.494+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9998158770 con 0x7f999809a3a0 2023-12-04T05:39:50.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.494+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994014820 con 0x7f999809a3a0 2023-12-04T05:39:50.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.494+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99981f0750 con 0x7f999809a3a0 2023-12-04T05:39:50.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.495+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99980671a0 con 0x7f999809a3a0 2023-12-04T05:39:50.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.495+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9994014980 con 0x7f999809a3a0 2023-12-04T05:39:50.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.495+0000 7f998e7fc640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f997c06b2a0 0x7f997c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.496+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f999408aae0 con 0x7f999809a3a0 2023-12-04T05:39:50.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.496+0000 7f999d1cc640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f997c06b2a0 0x7f997c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.496+0000 7f999d1cc640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f997c06b2a0 0x7f997c06d760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9988007fd0 tx=0x7f9988007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.500+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9994055950 con 0x7f999809a3a0 2023-12-04T05:39:50.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.666+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9998067460 con 0x7f999809a3a0 2023-12-04T05:39:50.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.666+0000 7f998e7fc640 1 -- 172.21.15.111:0/1847216210 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99940180a0 con 0x7f999809a3a0 2023-12-04T05:39:50.668 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f997c06b2a0 msgr2=0x7f997c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f997c06b2a0 0x7f997c06d760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9988007fd0 tx=0x7f9988007960 comp rx=0 tx=0).stop 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 msgr2=0x7f99981f0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 0x7f99981f0210 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f999400efc0 tx=0x7f999400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 shutdown_connections 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f997c06b2a0 0x7f997c06d760 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f999815e770 0x7f99980b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99980cc120 0x7f99980c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 --2- 172.21.15.111:0/1847216210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f999809a3a0 0x7f99981f0210 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 >> 172.21.15.111:0/1847216210 conn(0x7f99980b2ad0 msgr2=0x7f99980ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 shutdown_connections 2023-12-04T05:39:50.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.668+0000 7f999fc31640 1 -- 172.21.15.111:0/1847216210 wait complete. 2023-12-04T05:39:50.681 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.779+0000 7f2b7d79e640 1 Processor -- start 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.779+0000 7f2b7d79e640 1 -- start start 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b7d79e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2b78146a20 0x7f2b780b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b7d79e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b7806a4c0 0x7f2b7806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b7d79e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7815db90 con 0x7f2b78071500 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b7d79e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7806ae70 con 0x7f2b78146a20 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b7d79e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b3650 con 0x7f2b7806a4c0 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b7806a4c0 0x7f2b7806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2b78146a20 0x7f2b780b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2b78146a20 0x7f2b780b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46558/0 (socket says 172.21.15.111:46558) 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b777fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b7806a4c0 0x7f2b7806a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:36828/0 (socket says 172.21.15.111:36828) 2023-12-04T05:39:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b767fc640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f2b78071500 0x7f2b7815e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50760/0 (socket says 172.21.15.111:50760) 2023-12-04T05:39:50.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 -- 172.21.15.111:0/554880022 learned_addr learned my addr 172.21.15.111:0/554880022 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 -- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b7806a4c0 msgr2=0x7f2b7806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 --2- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b7806a4c0 0x7f2b7806a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 -- 172.21.15.111:0/554880022 >> v1:172.21.15.192:6789/0 conn(0x7f2b78071500 legacy=0x7f2b7815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.780+0000 7f2b76ffd640 1 -- 172.21.15.111:0/554880022 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7815d840 con 0x7f2b78146a20 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.781+0000 7f2b76ffd640 1 --2- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78146a20 0x7f2b780b3110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009e50 tx=0x7f2b6802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13670efa5874d977 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.781+0000 7f2b75ffb640 1 -- 172.21.15.111:0/554880022 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6802ca70 con 0x7f2b78146a20 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.781+0000 7f2b75ffb640 1 -- 172.21.15.111:0/554880022 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6802cbd0 con 0x7f2b78146a20 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.781+0000 7f2b75ffb640 1 -- 172.21.15.111:0/554880022 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b680343f0 con 0x7f2b78146a20 2023-12-04T05:39:50.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 -- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78146a20 msgr2=0x7f2b780b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78146a20 0x7f2b780b3110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009e50 tx=0x7f2b6802bed0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 -- 172.21.15.111:0/554880022 shutdown_connections 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b7806a4c0 0x7f2b7806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/554880022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78146a20 0x7f2b780b3110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 -- 172.21.15.111:0/554880022 >> 172.21.15.111:0/554880022 conn(0x7f2b7806b5d0 msgr2=0x7f2b7806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 -- 172.21.15.111:0/554880022 shutdown_connections 2023-12-04T05:39:50.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.782+0000 7f2b7d79e640 1 -- 172.21.15.111:0/554880022 wait complete. 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.783+0000 7f2b7d79e640 1 Processor -- start 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.783+0000 7f2b7d79e640 1 -- start start 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b7d79e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 0x7f2b78140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b7d79e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78071500 0x7f2b781411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b7d79e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b78146a20 0x7f2b780aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b7d79e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78141720 con 0x7f2b7806a4c0 2023-12-04T05:39:50.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b7d79e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78141930 con 0x7f2b78071500 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b7d79e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78141c10 con 0x7f2b78146a20 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 0x7f2b78140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 0x7f2b78140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36140/0 (socket says 172.21.15.111:36140) 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 -- 172.21.15.111:0/2725726259 learned_addr learned my addr 172.21.15.111:0/2725726259 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b777fe640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b78146a20 0x7f2b780aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b767fc640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78071500 0x7f2b781411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 -- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b78146a20 msgr2=0x7f2b780aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b78146a20 0x7f2b780aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 -- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78071500 msgr2=0x7f2b781411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78071500 0x7f2b781411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b76ffd640 1 -- 172.21.15.111:0/2725726259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68009b30 con 0x7f2b7806a4c0 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.784+0000 7f2b777fe640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b78146a20 0x7f2b780aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.785+0000 7f2b76ffd640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 0x7f2b78140bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b68000f80 tx=0x7f2b68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.785+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68009510 con 0x7f2b7806a4c0 2023-12-04T05:39:50.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.785+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6803e450 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.785+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b78141ef0 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.785+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6803e800 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.785+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b781424b0 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.786+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b780693d0 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.786+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f2b6803e960 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.787+0000 7f2b5f7fe640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2b6406b2a0 0x7f2b6406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.787+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b680ac910 con 0x7f2b7806a4c0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.787+0000 7f2b767fc640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2b6406b2a0 0x7f2b6406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:50.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.787+0000 7f2b767fc640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2b6406b2a0 0x7f2b6406d760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c007fd0 tx=0x7f2b6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:50.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.791+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b68077790 con 0x7f2b7806a4c0 2023-12-04T05:39:50.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.970+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b7806d870 con 0x7f2b7806a4c0 2023-12-04T05:39:50.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.970+0000 7f2b5f7fe640 1 -- 172.21.15.111:0/2725726259 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b6807adb0 con 0x7f2b7806a4c0 2023-12-04T05:39:50.971 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:50.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2b6406b2a0 msgr2=0x7f2b6406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2b6406b2a0 0x7f2b6406d760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c007fd0 tx=0x7f2b6c009040 comp rx=0 tx=0).stop 2023-12-04T05:39:50.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 msgr2=0x7f2b78140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:50.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 0x7f2b78140bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b68000f80 tx=0x7f2b68004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 shutdown_connections 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2b6406b2a0 0x7f2b6406d760 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b78146a20 0x7f2b780aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b78071500 0x7f2b781411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 --2- 172.21.15.111:0/2725726259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b7806a4c0 0x7f2b78140bc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 >> 172.21.15.111:0/2725726259 conn(0x7f2b7806b5d0 msgr2=0x7f2b78071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 shutdown_connections 2023-12-04T05:39:50.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:50.972+0000 7f2b7d79e640 1 -- 172.21.15.111:0/2725726259 wait complete. 2023-12-04T05:39:50.984 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-04T05:39:50.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:50.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-04T05:39:51.017 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:39:51.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:39:51.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:51.017 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-04T05:39:51.177 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:39:51.177 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:51.193 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:39:51.193 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:51.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:51.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 Processor -- start 2023-12-04T05:39:51.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 -- start start 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f029813cee0 0x7f029806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 0x7f029806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02980aea20 con 0x7f029815a490 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029813d2c0 con 0x7f029813cee0 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.388+0000 7f029cc9b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029815b3a0 con 0x7f0298140200 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0296d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f029815a490 0x7f0298071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54800/0 (socket says 172.21.15.111:54800) 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0296d9d640 1 -- 172.21.15.111:0/326794215 learned_addr learned my addr 172.21.15.111:0/326794215 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f028ffff640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 0x7f029806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637523346 0 0) 0x7f02980aea20 con 0x7f029815a490 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f029659c640 1 --2- 172.21.15.111:0/326794215 >> v2:172.21.15.192:3300/0 conn(0x7f029813cee0 0x7f029806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0284005730 con 0x7f029815a490 2023-12-04T05:39:51.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639943617 0 0) 0x7f0284005730 con 0x7f029815a490 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 msgr2=0x7f029806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 0x7f029806c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 >> v2:172.21.15.192:3300/0 conn(0x7f029813cee0 msgr2=0x7f029806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 --2- 172.21.15.111:0/326794215 >> v2:172.21.15.192:3300/0 conn(0x7f029813cee0 0x7f029806c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029815b0c0 con 0x7f029815a490 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f02900019f0 con 0x7f029815a490 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 >> v1:172.21.15.111:6789/0 conn(0x7f029815a490 legacy=0x7f0298071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0284005b80 0x7f0284005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02840069d0 0x7f0284006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 0x7f0284007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029813d2c0 con 0x7f0284005b80 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029815b3a0 con 0x7f02840069d0 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028400db90 con 0x7f0298140200 2023-12-04T05:39:51.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f029659c640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0284005b80 0x7f0284005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f0296d9d640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 0x7f0284007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.389+0000 7f028ffff640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02840069d0 0x7f0284006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f029659c640 1 -- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 msgr2=0x7f0284007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f029659c640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0298140200 0x7f0284007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f029659c640 1 -- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02840069d0 msgr2=0x7f0284006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f029659c640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02840069d0 0x7f0284006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f029659c640 1 -- 172.21.15.111:0/326794215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284005730 con 0x7f0284005b80 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f029659c640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0284005b80 0x7f0284005fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0288009af0 tx=0x7f0284007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf252b8cc60a8f04 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028802c710 con 0x7f0284005b80 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028802c870 con 0x7f0284005b80 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.390+0000 7f0295d9b640 1 -- 172.21.15.111:0/326794215 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028802cc00 con 0x7f0284005b80 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f029cc9b640 1 -- 172.21.15.111:0/326794215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0284005b80 msgr2=0x7f0284005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f029cc9b640 1 --2- 172.21.15.111:0/326794215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0284005b80 0x7f0284005fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0288009af0 tx=0x7f0284007a00 comp rx=0 tx=0).stop 2023-12-04T05:39:51.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f0296d9d640 1 -- 172.21.15.111:0/326794215 reap_dead start 2023-12-04T05:39:51.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f029cc9b640 1 -- 172.21.15.111:0/326794215 shutdown_connections 2023-12-04T05:39:51.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f029cc9b640 1 -- 172.21.15.111:0/326794215 >> 172.21.15.111:0/326794215 conn(0x7f029806a4c0 msgr2=0x7f029806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:51.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f029cc9b640 1 -- 172.21.15.111:0/326794215 shutdown_connections 2023-12-04T05:39:51.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.391+0000 7f029cc9b640 1 -- 172.21.15.111:0/326794215 wait complete. 2023-12-04T05:39:51.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.392+0000 7f029cc9b640 1 Processor -- start 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.392+0000 7f029cc9b640 1 -- start start 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029cc9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0284005b80 0x7f0298071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029cc9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 0x7f02981ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029cc9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02980add70 0x7f029813e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029cc9b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980723a0 con 0x7f0298138420 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029cc9b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298071b00 con 0x7f02980add70 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029cc9b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0298071de0 con 0x7f0284005b80 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 0x7f02981ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 0x7f02981ef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36168/0 (socket says 172.21.15.111:36168) 2023-12-04T05:39:51.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 -- 172.21.15.111:0/775118419 learned_addr learned my addr 172.21.15.111:0/775118419 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f0296d9d640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02980add70 0x7f029813e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f029659c640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0284005b80 0x7f0298071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 -- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0284005b80 msgr2=0x7f0298071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0284005b80 0x7f0298071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 -- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02980add70 msgr2=0x7f029813e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02980add70 0x7f029813e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 -- 172.21.15.111:0/775118419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980720c0 con 0x7f0298138420 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f0296d9d640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02980add70 0x7f029813e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.393+0000 7f028ffff640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 0x7f02981ef0b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0280011600 tx=0x7f0280011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.394+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000f920 con 0x7f0298138420 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.394+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028000fa80 con 0x7f0298138420 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.394+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000fdf0 con 0x7f0298138420 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.394+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0298138b70 con 0x7f0298138420 2023-12-04T05:39:51.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.394+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029813e9f0 con 0x7f0298138420 2023-12-04T05:39:51.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.395+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0298139100 con 0x7f0298138420 2023-12-04T05:39:51.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.395+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0280015460 con 0x7f0298138420 2023-12-04T05:39:51.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.395+0000 7f028effd640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f027806b2d0 0x7f027806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.395+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f028008a020 con 0x7f0298138420 2023-12-04T05:39:51.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.396+0000 7f029659c640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f027806b2d0 0x7f027806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.396+0000 7f029659c640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f027806b2d0 0x7f027806d790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0288036000 tx=0x7f0288030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.400+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0298139100 con 0x7f0298138420 2023-12-04T05:39:51.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.569+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0298139100 con 0x7f0298138420 2023-12-04T05:39:51.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.571+0000 7f028effd640 1 -- 172.21.15.111:0/775118419 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f0298139100 con 0x7f0298138420 2023-12-04T05:39:51.572 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:51.572 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f027806b2d0 msgr2=0x7f027806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f027806b2d0 0x7f027806d790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0288036000 tx=0x7f0288030000 comp rx=0 tx=0).stop 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 msgr2=0x7f02981ef0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 0x7f02981ef0b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0280011600 tx=0x7f0280011ac0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 shutdown_connections 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f027806b2d0 0x7f027806d790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02980add70 0x7f029813e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0298138420 0x7f02981ef0b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.572+0000 7f029cc9b640 1 --2- 172.21.15.111:0/775118419 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0284005b80 0x7f0298071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.573+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 >> 172.21.15.111:0/775118419 conn(0x7f029806a4c0 msgr2=0x7f0298150b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:51.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.573+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 shutdown_connections 2023-12-04T05:39:51.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.573+0000 7f029cc9b640 1 -- 172.21.15.111:0/775118419 wait complete. 2023-12-04T05:39:51.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:51.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:51.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.686+0000 7f11e9190640 1 Processor -- start 2023-12-04T05:39:51.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.686+0000 7f11e9190640 1 -- start start 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e9190640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f11e415d480 0x7f11e40d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e9190640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e40d01e0 0x7f11e4160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e9190640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e415ddc0 con 0x7f11e40d1310 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e9190640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e415e0a0 con 0x7f11e415d480 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e9190640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e415e380 con 0x7f11e40d01e0 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e2d9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f11e415d480 0x7f11e40d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e40d01e0 0x7f11e4160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e259c640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f11e40d1310 0x7f11e40cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50770/0 (socket says 172.21.15.111:50770) 2023-12-04T05:39:51.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e259c640 1 -- 172.21.15.111:0/887308981 learned_addr learned my addr 172.21.15.111:0/887308981 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:51.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.687+0000 7f11e2d9d640 1 --2- 172.21.15.111:0/887308981 >> v2:172.21.15.192:3300/0 conn(0x7f11e415d480 0x7f11e40d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46590/0 (socket says 172.21.15.111:46590) 2023-12-04T05:39:51.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.688+0000 7f11e2d9d640 1 -- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e40d01e0 msgr2=0x7f11e4160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:51.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.688+0000 7f11e2d9d640 1 --2- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e40d01e0 0x7f11e4160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.688+0000 7f11e2d9d640 1 -- 172.21.15.111:0/887308981 >> v1:172.21.15.192:6789/0 conn(0x7f11e40d1310 legacy=0x7f11e40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.688+0000 7f11e2d9d640 1 -- 172.21.15.111:0/887308981 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e415e660 con 0x7f11e415d480 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e2d9d640 1 --2- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e415d480 0x7f11e40d0dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009d90 tx=0x7f11d402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df958db428e6f89 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e1d9b640 1 -- 172.21.15.111:0/887308981 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d402cc50 con 0x7f11e415d480 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e1d9b640 1 -- 172.21.15.111:0/887308981 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d402cdb0 con 0x7f11e415d480 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e1d9b640 1 -- 172.21.15.111:0/887308981 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4034660 con 0x7f11e415d480 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 -- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e415d480 msgr2=0x7f11e40d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 --2- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e415d480 0x7f11e40d0dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009d90 tx=0x7f11d402bea0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 -- 172.21.15.111:0/887308981 shutdown_connections 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 --2- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e40d01e0 0x7f11e4160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 --2- 172.21.15.111:0/887308981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e415d480 0x7f11e40d0dd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 -- 172.21.15.111:0/887308981 >> 172.21.15.111:0/887308981 conn(0x7f11e409c540 msgr2=0x7f11e40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 -- 172.21.15.111:0/887308981 shutdown_connections 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.689+0000 7f11e9190640 1 -- 172.21.15.111:0/887308981 wait complete. 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.691+0000 7f11e9190640 1 Processor -- start 2023-12-04T05:39:51.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.691+0000 7f11e9190640 1 -- start start 2023-12-04T05:39:51.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.691+0000 7f11e9190640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 0x7f11e40b8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.691+0000 7f11e9190640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e40d1310 0x7f11e4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e9190640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e415d480 0x7f11e4159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e9190640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40b7eb0 con 0x7f11e40d01e0 2023-12-04T05:39:51.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e9190640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40b72a0 con 0x7f11e40d1310 2023-12-04T05:39:51.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e9190640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40b7580 con 0x7f11e415d480 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 0x7f11e40b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 0x7f11e40b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36180/0 (socket says 172.21.15.111:36180) 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 -- 172.21.15.111:0/3596184112 learned_addr learned my addr 172.21.15.111:0/3596184112 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e259c640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e40d1310 0x7f11e4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e359e640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e415d480 0x7f11e4159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 -- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e415d480 msgr2=0x7f11e4159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e415d480 0x7f11e4159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 -- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e40d1310 msgr2=0x7f11e4156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e40d1310 0x7f11e4156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 -- 172.21.15.111:0/3596184112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4009a70 con 0x7f11e40d01e0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e2d9d640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 0x7f11e40b8340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009eb0 tx=0x7f11d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d403e610 con 0x7f11e40d01e0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d403e770 con 0x7f11e40d01e0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d403eae0 con 0x7f11e40d01e0 2023-12-04T05:39:51.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e40b7860 con 0x7f11e40d01e0 2023-12-04T05:39:51.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.692+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e41566a0 con 0x7f11e40d01e0 2023-12-04T05:39:51.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.693+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e4067980 con 0x7f11e40d01e0 2023-12-04T05:39:51.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.694+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f11d403b050 con 0x7f11e40d01e0 2023-12-04T05:39:51.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.694+0000 7f11cb7fe640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f11d006b2a0 0x7f11d006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.694+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11d40ac4a0 con 0x7f11e40d01e0 2023-12-04T05:39:51.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.694+0000 7f11e259c640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f11d006b2a0 0x7f11d006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.694+0000 7f11e259c640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f11d006b2a0 0x7f11d006d760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11e40ca170 tx=0x7f11d8007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.698+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11d40773c0 con 0x7f11e40d01e0 2023-12-04T05:39:51.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.864+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11e4069360 con 0x7f11e40d01e0 2023-12-04T05:39:51.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.865+0000 7f11cb7fe640 1 -- 172.21.15.111:0/3596184112 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11d407fe60 con 0x7f11e40d01e0 2023-12-04T05:39:51.866 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f11d006b2a0 msgr2=0x7f11d006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f11d006b2a0 0x7f11d006d760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11e40ca170 tx=0x7f11d8007e80 comp rx=0 tx=0).stop 2023-12-04T05:39:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 msgr2=0x7f11e40b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 0x7f11e40b8340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009eb0 tx=0x7f11d4004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 shutdown_connections 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f11d006b2a0 0x7f11d006d760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e415d480 0x7f11e4159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e40d1310 0x7f11e4156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 --2- 172.21.15.111:0/3596184112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40d01e0 0x7f11e40b8340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 >> 172.21.15.111:0/3596184112 conn(0x7f11e409c540 msgr2=0x7f11e415f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 shutdown_connections 2023-12-04T05:39:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.867+0000 7f11e9190640 1 -- 172.21.15.111:0/3596184112 wait complete. 2023-12-04T05:39:51.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:51.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.979+0000 7fb67fd6f640 1 Processor -- start 2023-12-04T05:39:51.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.980+0000 7fb67fd6f640 1 -- start start 2023-12-04T05:39:51.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.980+0000 7fb67fd6f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb67806f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.980+0000 7fb67fd6f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb6780cc3a0 0x7fb6780cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.980+0000 7fb67fd6f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6780cfd60 con 0x7fb678070110 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.980+0000 7fb67fd6f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67809a5d0 con 0x7fb6780cc3a0 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.980+0000 7fb67fd6f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67809a8b0 con 0x7fb678099a00 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67db0b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb67806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67db0b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb67806f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:36870/0 (socket says 172.21.15.111:36870) 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67e30c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb6780cc3a0 0x7fb6780cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67db0b640 1 -- 172.21.15.111:0/2177781537 learned_addr learned my addr 172.21.15.111:0/2177781537 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67e30c640 1 -- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 msgr2=0x7fb67806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67e30c640 1 --2- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb67806f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67e30c640 1 -- 172.21.15.111:0/2177781537 >> v1:172.21.15.192:6789/0 conn(0x7fb678070110 legacy=0x7fb678099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.981+0000 7fb67e30c640 1 -- 172.21.15.111:0/2177781537 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780cf560 con 0x7fb6780cc3a0 2023-12-04T05:39:51.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.982+0000 7fb67e30c640 1 --2- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780cc830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb6700097a0 tx=0x7fb67002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35bb04b23ba0b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.982+0000 7fb67cb09640 1 -- 172.21.15.111:0/2177781537 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670031830 con 0x7fb6780cc3a0 2023-12-04T05:39:51.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.982+0000 7fb67cb09640 1 -- 172.21.15.111:0/2177781537 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670031990 con 0x7fb6780cc3a0 2023-12-04T05:39:51.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.982+0000 7fb67cb09640 1 -- 172.21.15.111:0/2177781537 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670031d20 con 0x7fb6780cc3a0 2023-12-04T05:39:51.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 -- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 msgr2=0x7fb6780cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780cc830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb6700097a0 tx=0x7fb67002b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 -- 172.21.15.111:0/2177781537 shutdown_connections 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780cc830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/2177781537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb67806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 -- 172.21.15.111:0/2177781537 >> 172.21.15.111:0/2177781537 conn(0x7fb6780b2a70 msgr2=0x7fb6780b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 -- 172.21.15.111:0/2177781537 shutdown_connections 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.983+0000 7fb67fd6f640 1 -- 172.21.15.111:0/2177781537 wait complete. 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.984+0000 7fb67fd6f640 1 Processor -- start 2023-12-04T05:39:51.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.984+0000 7fb67fd6f640 1 -- start start 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 0x7fb678156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb6780b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67815a890 con 0x7fb678070110 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb678156ab0 con 0x7fb6780cc3a0 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780c5a30 con 0x7fb678099a00 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 0x7fb678156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 0x7fb678156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36184/0 (socket says 172.21.15.111:36184) 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 -- 172.21.15.111:0/910681323 learned_addr learned my addr 172.21.15.111:0/910681323 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67d30a640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb6780b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67e30c640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 -- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 msgr2=0x7fb6780b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb6780b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 -- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 msgr2=0x7fb6780b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 -- 172.21.15.111:0/910681323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb670009400 con 0x7fb678070110 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67d30a640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb6780b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67db0b640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 0x7fb678156570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400cda0 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400f040 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6780c5d10 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674014680 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.985+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6780c62d0 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.986+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb678069360 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.987+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb6740147e0 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.987+0000 7fb66e7fc640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb65c06b2a0 0x7fb65c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.987+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb67408a370 con 0x7fb678070110 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.987+0000 7fb67d30a640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb65c06b2a0 0x7fb65c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:51.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.988+0000 7fb67d30a640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb65c06b2a0 0x7fb65c06d760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb6780c6ec0 tx=0x7fb668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:51.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:51.991+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6740551e0 con 0x7fb678070110 2023-12-04T05:39:52.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.158+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6780cee80 con 0x7fb678070110 2023-12-04T05:39:52.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.158+0000 7fb66e7fc640 1 -- 172.21.15.111:0/910681323 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb674058800 con 0x7fb678070110 2023-12-04T05:39:52.160 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb65c06b2a0 msgr2=0x7fb65c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb65c06b2a0 0x7fb65c06d760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb6780c6ec0 tx=0x7fb668009040 comp rx=0 tx=0).stop 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 msgr2=0x7fb678156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 0x7fb678156570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 shutdown_connections 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb65c06b2a0 0x7fb65c06d760 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6780cc3a0 0x7fb6780b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb678099a00 0x7fb6780b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 --2- 172.21.15.111:0/910681323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb678070110 0x7fb678156570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 >> 172.21.15.111:0/910681323 conn(0x7fb6780b2a70 msgr2=0x7fb6780cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:52.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 shutdown_connections 2023-12-04T05:39:52.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.160+0000 7fb67fd6f640 1 -- 172.21.15.111:0/910681323 wait complete. 2023-12-04T05:39:52.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:52.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.274+0000 7f45a8943640 1 Processor -- start 2023-12-04T05:39:52.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.274+0000 7f45a8943640 1 -- start start 2023-12-04T05:39:52.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a8943640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 0x7f45a40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a8943640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45a4070ff0 0x7f45a40714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a8943640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40cd790 con 0x7f45a40cd020 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a8943640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a4159c20 con 0x7f45a4070ff0 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a8943640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40ced70 con 0x7f45a4070c10 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a259c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 0x7f45a40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a2d9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45a4070ff0 0x7f45a40714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a259c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 0x7f45a40ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:36896/0 (socket says 172.21.15.111:36896) 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a259c640 1 -- 172.21.15.111:0/3338233054 learned_addr learned my addr 172.21.15.111:0/3338233054 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.275+0000 7f45a1d9b640 1 --1- 172.21.15.111:0/3338233054 >> v1:172.21.15.192:6789/0 conn(0x7f45a40cd020 0x7f45a40b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50794/0 (socket says 172.21.15.111:50794) 2023-12-04T05:39:52.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.276+0000 7f45a2d9d640 1 -- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 msgr2=0x7f45a40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.276+0000 7f45a2d9d640 1 --2- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 0x7f45a40ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.276+0000 7f45a2d9d640 1 -- 172.21.15.111:0/3338233054 >> v1:172.21.15.192:6789/0 conn(0x7f45a40cd020 legacy=0x7f45a40b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.276+0000 7f45a2d9d640 1 -- 172.21.15.111:0/3338233054 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40cda70 con 0x7f45a4070ff0 2023-12-04T05:39:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.276+0000 7f45a259c640 1 --2- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 0x7f45a40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.276+0000 7f45a2d9d640 1 --2- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a4070ff0 0x7f45a40714a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f459c0097a0 tx=0x7f459c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f165b5a2535c7239 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a159a640 1 -- 172.21.15.111:0/3338233054 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c031830 con 0x7f45a4070ff0 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a159a640 1 -- 172.21.15.111:0/3338233054 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c031990 con 0x7f45a4070ff0 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a159a640 1 -- 172.21.15.111:0/3338233054 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c031d20 con 0x7f45a4070ff0 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 -- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a4070ff0 msgr2=0x7f45a40714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 --2- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a4070ff0 0x7f45a40714a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f459c0097a0 tx=0x7f459c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 -- 172.21.15.111:0/3338233054 shutdown_connections 2023-12-04T05:39:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 --2- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a4070ff0 0x7f45a40714a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 --2- 172.21.15.111:0/3338233054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070c10 0x7f45a40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 -- 172.21.15.111:0/3338233054 >> 172.21.15.111:0/3338233054 conn(0x7f45a40b0cf0 msgr2=0x7f45a40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 -- 172.21.15.111:0/3338233054 shutdown_connections 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.277+0000 7f45a8943640 1 -- 172.21.15.111:0/3338233054 wait complete. 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.278+0000 7f45a8943640 1 Processor -- start 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 -- start start 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 0x7f45a41f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070ff0 0x7f45a415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a40cd020 0x7f45a40afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a415b0c0 con 0x7f45a4070c10 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4157560 con 0x7f45a40cd020 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a8943640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4158ac0 con 0x7f45a4070ff0 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 0x7f45a41f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 0x7f45a41f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36198/0 (socket says 172.21.15.111:36198) 2023-12-04T05:39:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.279+0000 7f45a259c640 1 -- 172.21.15.111:0/2282697357 learned_addr learned my addr 172.21.15.111:0/2282697357 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a1d9b640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070ff0 0x7f45a415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a2d9d640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a40cd020 0x7f45a40afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a259c640 1 -- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070ff0 msgr2=0x7f45a415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a259c640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070ff0 0x7f45a415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a259c640 1 -- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a40cd020 msgr2=0x7f45a40afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a259c640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a40cd020 0x7f45a40afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a259c640 1 -- 172.21.15.111:0/2282697357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594009b30 con 0x7f45a4070c10 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a259c640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 0x7f45a41f60a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f459400efc0 tx=0x7f459400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594014480 con 0x7f45a4070c10 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400f040 con 0x7f45a4070c10 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c009400 con 0x7f45a4070c10 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940147e0 con 0x7f45a4070c10 2023-12-04T05:39:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.280+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a4158190 con 0x7f45a4070c10 2023-12-04T05:39:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.281+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a4069360 con 0x7f45a4070c10 2023-12-04T05:39:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.281+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4594014940 con 0x7f45a4070c10 2023-12-04T05:39:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.282+0000 7f4592ffd640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458406b2a0 0x7f458406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.282+0000 7f45a1d9b640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458406b2a0 0x7f458406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.282+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f459408a540 con 0x7f45a4070c10 2023-12-04T05:39:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.282+0000 7f45a1d9b640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458406b2a0 0x7f458406d760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45a40b0930 tx=0x7f4598009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.286+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45940553b0 con 0x7f45a4070c10 2023-12-04T05:39:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.454+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45a40724d0 con 0x7f45a4070c10 2023-12-04T05:39:52.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.454+0000 7f4592ffd640 1 -- 172.21.15.111:0/2282697357 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45940589d0 con 0x7f45a4070c10 2023-12-04T05:39:52.456 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458406b2a0 msgr2=0x7f458406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458406b2a0 0x7f458406d760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45a40b0930 tx=0x7f4598009040 comp rx=0 tx=0).stop 2023-12-04T05:39:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 msgr2=0x7f45a41f60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 0x7f45a41f60a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f459400efc0 tx=0x7f459400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 shutdown_connections 2023-12-04T05:39:52.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458406b2a0 0x7f458406d760 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a40cd020 0x7f45a40afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45a4070ff0 0x7f45a415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 --2- 172.21.15.111:0/2282697357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45a4070c10 0x7f45a41f60a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.456+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 >> 172.21.15.111:0/2282697357 conn(0x7f45a40b0cf0 msgr2=0x7f45a4071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:52.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.457+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 shutdown_connections 2023-12-04T05:39:52.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.457+0000 7f45a8943640 1 -- 172.21.15.111:0/2282697357 wait complete. 2023-12-04T05:39:52.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:52.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 Processor -- start 2023-12-04T05:39:52.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 -- start start 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00440d1780 0x7f00440d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440b2ad0 0x7f00440ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00440cdcb0 con 0x7f00440b20a0 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00440d39e0 con 0x7f00440d1780 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.570+0000 7f0049897640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00440d3cc0 con 0x7f00440b2ad0 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f003a7fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f00440b20a0 0x7f00440d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54816/0 (socket says 172.21.15.111:54816) 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f003a7fc640 1 -- 172.21.15.111:0/2980415918 learned_addr learned my addr 172.21.15.111:0/2980415918 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f0042ffd640 1 --2- 172.21.15.111:0/2980415918 >> v2:172.21.15.192:3300/0 conn(0x7f00440d1780 0x7f00440d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f00437fe640 1 --2- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440b2ad0 0x7f00440ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f00427fc640 1 -- 172.21.15.111:0/2980415918 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030708331 0 0) 0x7f00440cdcb0 con 0x7f00440b20a0 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f00427fc640 1 -- 172.21.15.111:0/2980415918 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0030003520 con 0x7f00440b20a0 2023-12-04T05:39:52.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f0042ffd640 1 -- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440b2ad0 msgr2=0x7f00440ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f0042ffd640 1 --2- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440b2ad0 0x7f00440ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f0042ffd640 1 -- 172.21.15.111:0/2980415918 >> v1:172.21.15.111:6789/0 conn(0x7f00440b20a0 legacy=0x7f00440d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f00427fc640 1 -- 172.21.15.111:0/2980415918 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230946592 0 0) 0x7f0030003520 con 0x7f00440b20a0 2023-12-04T05:39:52.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.571+0000 7f0042ffd640 1 -- 172.21.15.111:0/2980415918 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00440cdf50 con 0x7f00440d1780 2023-12-04T05:39:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.572+0000 7f0042ffd640 1 --2- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440d1780 0x7f00440d01e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00340097a0 tx=0x7f003402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e140d00f47378a86 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.572+0000 7f00427fc640 1 -- 172.21.15.111:0/2980415918 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c001670 con 0x7f00440d1780 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.572+0000 7f00427fc640 1 -- 172.21.15.111:0/2980415918 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003402e050 con 0x7f00440d1780 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.572+0000 7f00427fc640 1 -- 172.21.15.111:0/2980415918 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034030710 con 0x7f00440d1780 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 -- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440d1780 msgr2=0x7f00440d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 --2- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440d1780 0x7f00440d01e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00340097a0 tx=0x7f003402b710 comp rx=0 tx=0).stop 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 -- 172.21.15.111:0/2980415918 shutdown_connections 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 --2- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440b2ad0 0x7f00440ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 --2- 172.21.15.111:0/2980415918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440d1780 0x7f00440d01e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 -- 172.21.15.111:0/2980415918 >> 172.21.15.111:0/2980415918 conn(0x7f004409c4e0 msgr2=0x7f00440d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 -- 172.21.15.111:0/2980415918 shutdown_connections 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.573+0000 7f0049897640 1 -- 172.21.15.111:0/2980415918 wait complete. 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.574+0000 7f0049897640 1 Processor -- start 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.574+0000 7f0049897640 1 -- start start 2023-12-04T05:39:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 0x7f00441511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440b2ad0 0x7f0044160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440d1780 0x7f00440cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00440cd680 con 0x7f00440b20a0 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004415f820 con 0x7f00440b2ad0 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004415ff20 con 0x7f00440d1780 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 0x7f00441511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 0x7f00441511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36202/0 (socket says 172.21.15.111:36202) 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 -- 172.21.15.111:0/1565042641 learned_addr learned my addr 172.21.15.111:0/1565042641 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f00437fe640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440d1780 0x7f00440cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f003a7fc640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440b2ad0 0x7f0044160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 -- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440d1780 msgr2=0x7f00440cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440d1780 0x7f00440cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 -- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440b2ad0 msgr2=0x7f0044160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440b2ad0 0x7f0044160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 -- 172.21.15.111:0/1565042641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034009400 con 0x7f00440b20a0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f003a7fc640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440b2ad0 0x7f0044160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0042ffd640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 0x7f00441511d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0034000c00 tx=0x7f00340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003403c620 con 0x7f00440b20a0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003403c7a0 con 0x7f00440b20a0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0044160200 con 0x7f00440b20a0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003403cb50 con 0x7f00440b20a0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.575+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004415fb00 con 0x7f00440b20a0 2023-12-04T05:39:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.576+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00441606c0 con 0x7f00440b20a0 2023-12-04T05:39:52.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.577+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0034039070 con 0x7f00440b20a0 2023-12-04T05:39:52.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.577+0000 7f0040ff9640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f002406b2a0 0x7f002406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.577+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00340ab390 con 0x7f00440b20a0 2023-12-04T05:39:52.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.577+0000 7f003a7fc640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f002406b2a0 0x7f002406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.578+0000 7f003a7fc640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f002406b2a0 0x7f002406d760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f00440cefa0 tx=0x7f002c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.581+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0034076200 con 0x7f00440b20a0 2023-12-04T05:39:52.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.759+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f00440693d0 con 0x7f00440b20a0 2023-12-04T05:39:52.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.759+0000 7f0040ff9640 1 -- 172.21.15.111:0/1565042641 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0034079820 con 0x7f00440b20a0 2023-12-04T05:39:52.760 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:52.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f002406b2a0 msgr2=0x7f002406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f002406b2a0 0x7f002406d760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f00440cefa0 tx=0x7f002c009040 comp rx=0 tx=0).stop 2023-12-04T05:39:52.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 msgr2=0x7f00441511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 0x7f00441511d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0034000c00 tx=0x7f00340046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 shutdown_connections 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f002406b2a0 0x7f002406d760 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00440d1780 0x7f00440cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00440b2ad0 0x7f0044160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 --2- 172.21.15.111:0/1565042641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00440b20a0 0x7f00441511d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 >> 172.21.15.111:0/1565042641 conn(0x7f004409c4e0 msgr2=0x7f004406b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 shutdown_connections 2023-12-04T05:39:52.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.761+0000 7f0049897640 1 -- 172.21.15.111:0/1565042641 wait complete. 2023-12-04T05:39:52.773 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:39:52.773 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:52.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:52.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.925+0000 7fb73b948640 1 Processor -- start 2023-12-04T05:39:52.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.926+0000 7fb73b948640 1 -- start start 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.926+0000 7fb73b948640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb73415f7b0 0x7fb734071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.926+0000 7fb73b948640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73415fc10 con 0x7fb734144430 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.926+0000 7fb73b948640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb734070bd0 con 0x7fb7340721b0 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.926+0000 7fb73b948640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb734144810 con 0x7fb73415f7b0 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb7396e4640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb734144430 0x7fb7340703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54828/0 (socket says 172.21.15.111:54828) 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb7396e4640 1 -- 172.21.15.111:0/2011889409 learned_addr learned my addr 172.21.15.111:0/2011889409 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069853861 0 0) 0x7fb73415fc10 con 0x7fb734144430 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb720003520 con 0x7fb734144430 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb738ee3640 1 --2- 172.21.15.111:0/2011889409 >> v2:172.21.15.192:3300/0 conn(0x7fb73415f7b0 0x7fb734071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2451192245 0 0) 0x7fb720003520 con 0x7fb734144430 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 >> v2:172.21.15.192:3300/0 conn(0x7fb73415f7b0 msgr2=0x7fb734071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 --2- 172.21.15.111:0/2011889409 >> v2:172.21.15.192:3300/0 conn(0x7fb73415f7b0 0x7fb734071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 >> v1:172.21.15.192:6789/0 conn(0x7fb7340721b0 legacy=0x7fb73406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734070eb0 con 0x7fb734144430 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fb724001670 con 0x7fb734144430 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb738ee3640 1 --2- 172.21.15.111:0/2011889409 >> v2:172.21.15.192:3300/0 conn(0x7fb73415f7b0 0x7fb734071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 >> v1:172.21.15.111:6789/0 conn(0x7fb734144430 legacy=0x7fb7340703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb720003970 0x7fb720003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7200047c0 0x7fb720004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb72000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734144810 con 0x7fb720005180 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72000c0b0 con 0x7fb720003970 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72000c270 con 0x7fb7200047c0 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb739ee5640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb72000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb7396e4640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb720003970 0x7fb720003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.927+0000 7fb738ee3640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7200047c0 0x7fb720004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 -- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7200047c0 msgr2=0x7fb720004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7200047c0 0x7fb720004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 -- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb720003970 msgr2=0x7fb720003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb720003970 0x7fb720003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 -- 172.21.15.111:0/2011889409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720003520 con 0x7fb720005180 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 -- 172.21.15.111:0/2011889409 reap_dead start 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb739ee5640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb72000b980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb734067fd0 tx=0x7fb7200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f38b4cacb53d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c006b50 con 0x7fb720005180 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c006cd0 con 0x7fb720005180 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.928+0000 7fb72bfff640 1 -- 172.21.15.111:0/2011889409 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c007260 con 0x7fb720005180 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 -- 172.21.15.111:0/2011889409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 msgr2=0x7fb72000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb72000b980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb734067fd0 tx=0x7fb7200057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 -- 172.21.15.111:0/2011889409 shutdown_connections 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 --2- 172.21.15.111:0/2011889409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb72000b980 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 -- 172.21.15.111:0/2011889409 >> 172.21.15.111:0/2011889409 conn(0x7fb73406a4c0 msgr2=0x7fb73406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 -- 172.21.15.111:0/2011889409 shutdown_connections 2023-12-04T05:39:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.929+0000 7fb73b948640 1 -- 172.21.15.111:0/2011889409 wait complete. 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.930+0000 7fb73b948640 1 Processor -- start 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.930+0000 7fb73b948640 1 -- start start 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb73b948640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb73415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb73b948640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7341f1670 0x7fb7340c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb73b948640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73413c0c0 0x7fb73413c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb73b948640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73415aeb0 con 0x7fb720005180 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb73b948640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340c5fa0 con 0x7fb7341f1670 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb73b948640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340c4000 con 0x7fb73413c0c0 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb73415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb73415a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36226/0 (socket says 172.21.15.111:36226) 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 -- 172.21.15.111:0/3809936961 learned_addr learned my addr 172.21.15.111:0/3809936961 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 -- 172.21.15.111:0/3809936961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73413c0c0 msgr2=0x7fb73413c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73413c0c0 0x7fb73413c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 -- 172.21.15.111:0/3809936961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7341f1670 msgr2=0x7fb7340c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7341f1670 0x7fb7340c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 -- 172.21.15.111:0/3809936961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340c42e0 con 0x7fb720005180 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.931+0000 7fb7396e4640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb73415a970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb724007290 tx=0x7fb724007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.932+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400f960 con 0x7fb720005180 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.932+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400fac0 con 0x7fb720005180 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.932+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340c45c0 con 0x7fb720005180 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.932+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400fe50 con 0x7fb720005180 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.932+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7340c6280 con 0x7fb720005180 2023-12-04T05:39:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.933+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7340c4800 con 0x7fb720005180 2023-12-04T05:39:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.933+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb72401d460 con 0x7fb720005180 2023-12-04T05:39:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.933+0000 7fb729ffb640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb71806b2d0 0x7fb71806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.933+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb724089d10 con 0x7fb720005180 2023-12-04T05:39:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.933+0000 7fb738ee3640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb71806b2d0 0x7fb71806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.934+0000 7fb738ee3640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb71806b2d0 0x7fb71806d790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb730006fa0 tx=0x7fb730006f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:52.938+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7340c4800 con 0x7fb720005180 2023-12-04T05:39:53.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.104+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7340c4800 con 0x7fb720005180 2023-12-04T05:39:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.105+0000 7fb729ffb640 1 -- 172.21.15.111:0/3809936961 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fb7240586a0 con 0x7fb720005180 2023-12-04T05:39:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:53.107 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb71806b2d0 msgr2=0x7fb71806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb71806b2d0 0x7fb71806d790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb730006fa0 tx=0x7fb730006f10 comp rx=0 tx=0).stop 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 msgr2=0x7fb73415a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb73415a970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb724007290 tx=0x7fb724007750 comp rx=0 tx=0).stop 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 shutdown_connections 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb71806b2d0 0x7fb71806d790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73413c0c0 0x7fb73413c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7341f1670 0x7fb7340c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 --2- 172.21.15.111:0/3809936961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb720005180 0x7fb73415a970 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 >> 172.21.15.111:0/3809936961 conn(0x7fb73406a4c0 msgr2=0x7fb73406c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 shutdown_connections 2023-12-04T05:39:53.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.107+0000 7fb73b948640 1 -- 172.21.15.111:0/3809936961 wait complete. 2023-12-04T05:39:53.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:53.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:53.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.221+0000 7f28abc23640 1 Processor -- start 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.222+0000 7f28abc23640 1 -- start start 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.222+0000 7f28abc23640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f28a40ae600 0x7f28a415da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.222+0000 7f28abc23640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 0x7f28a40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.222+0000 7f28abc23640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a415e190 con 0x7f28a40af750 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.222+0000 7f28abc23640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a4157f60 con 0x7f28a40ae600 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.222+0000 7f28abc23640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40cd190 con 0x7f28a40ac100 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a99bf640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f28a40af750 0x7f28a40afb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54840/0 (socket says 172.21.15.111:54840) 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a99bf640 1 -- 172.21.15.111:0/2144389990 learned_addr learned my addr 172.21.15.111:0/2144389990 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28aa1c0640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 0x7f28a40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a91be640 1 --2- 172.21.15.111:0/2144389990 >> v2:172.21.15.192:3300/0 conn(0x7f28a40ae600 0x7f28a415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554054271 0 0) 0x7f28a415e190 con 0x7f28a40af750 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2890003520 con 0x7f28a40af750 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48147605 0 0) 0x7f2890003520 con 0x7f28a40af750 2023-12-04T05:39:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 msgr2=0x7f28a40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 0x7f28a40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 >> v2:172.21.15.192:3300/0 conn(0x7f28a40ae600 msgr2=0x7f28a415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 --2- 172.21.15.111:0/2144389990 >> v2:172.21.15.192:3300/0 conn(0x7f28a40ae600 0x7f28a415da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a415e470 con 0x7f28a40af750 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f28a0001670 con 0x7f28a40af750 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 >> v1:172.21.15.111:6789/0 conn(0x7f28a40af750 legacy=0x7f28a40afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2890003970 0x7f2890003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28900047c0 0x7f2890004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 0x7f2890005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289000b980 con 0x7f2890003970 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289000bac0 con 0x7f28900047c0 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289000bc00 con 0x7f28a40ac100 2023-12-04T05:39:53.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2890003970 0x7f2890003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28a91be640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28900047c0 0x7f2890004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.223+0000 7f28aa1c0640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 0x7f2890005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a99bf640 1 -- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 msgr2=0x7f2890005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28a40ac100 0x7f2890005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a99bf640 1 -- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28900047c0 msgr2=0x7f2890004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28900047c0 0x7f2890004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a99bf640 1 -- 172.21.15.111:0/2144389990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890003520 con 0x7f2890003970 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2890003970 0x7f2890003d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28a000c270 tx=0x7f28900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8eaa70b878ebd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a002cae0 con 0x7f2890003970 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a002cc40 con 0x7f2890003970 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.224+0000 7f28a89bd640 1 -- 172.21.15.111:0/2144389990 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a00058d0 con 0x7f2890003970 2023-12-04T05:39:53.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28abc23640 1 -- 172.21.15.111:0/2144389990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2890003970 msgr2=0x7f2890003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28abc23640 1 --2- 172.21.15.111:0/2144389990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2890003970 0x7f2890003d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28a000c270 tx=0x7f28900057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28aa1c0640 1 -- 172.21.15.111:0/2144389990 reap_dead start 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28abc23640 1 -- 172.21.15.111:0/2144389990 shutdown_connections 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28abc23640 1 -- 172.21.15.111:0/2144389990 >> 172.21.15.111:0/2144389990 conn(0x7f28a40b4640 msgr2=0x7f28a40b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28abc23640 1 -- 172.21.15.111:0/2144389990 shutdown_connections 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.225+0000 7f28abc23640 1 -- 172.21.15.111:0/2144389990 wait complete. 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.226+0000 7f28abc23640 1 Processor -- start 2023-12-04T05:39:53.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.226+0000 7f28abc23640 1 -- start start 2023-12-04T05:39:53.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28abc23640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2890003970 0x7f28a41f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28abc23640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 0x7f28a40afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28abc23640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28a4160320 0x7f28a40b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28abc23640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4156110 con 0x7f28a4145840 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28abc23640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41f3710 con 0x7f28a4160320 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28abc23640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41f2bc0 con 0x7f2890003970 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 0x7f28a40afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 0x7f28a40afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36240/0 (socket says 172.21.15.111:36240) 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 -- 172.21.15.111:0/2134803913 learned_addr learned my addr 172.21.15.111:0/2134803913 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2890003970 0x7f28a41f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28aa1c0640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28a4160320 0x7f28a40b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 -- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2890003970 msgr2=0x7f28a41f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2890003970 0x7f28a41f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 -- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28a4160320 msgr2=0x7f28a40b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28a4160320 0x7f28a40b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 -- 172.21.15.111:0/2134803913 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a41f2e70 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.227+0000 7f28a91be640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 0x7f28a40afb30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2894011960 tx=0x7f2894011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.228+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289400faf0 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.228+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289400fc50 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.228+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a41f3150 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.228+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894014440 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.228+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a40c8b10 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.229+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a4069360 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.229+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f28940145a0 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.229+0000 7f289a7fc640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f288806b2a0 0x7f288806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.230+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f288806b2a0 0x7f288806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.230+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f289408a260 con 0x7f28a4145840 2023-12-04T05:39:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.230+0000 7f28a99bf640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f288806b2a0 0x7f288806d760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28a0034000 tx=0x7f28a0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.234+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2894055180 con 0x7f28a4145840 2023-12-04T05:39:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.402+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28a406b780 con 0x7f28a4145840 2023-12-04T05:39:53.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.402+0000 7f289a7fc640 1 -- 172.21.15.111:0/2134803913 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28940587a0 con 0x7f28a4145840 2023-12-04T05:39:53.404 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:53.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f288806b2a0 msgr2=0x7f288806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f288806b2a0 0x7f288806d760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28a0034000 tx=0x7f28a0007000 comp rx=0 tx=0).stop 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 msgr2=0x7f28a40afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 0x7f28a40afb30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2894011960 tx=0x7f2894011e20 comp rx=0 tx=0).stop 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 shutdown_connections 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f288806b2a0 0x7f288806d760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28a4160320 0x7f28a40b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28a4145840 0x7f28a40afb30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 --2- 172.21.15.111:0/2134803913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2890003970 0x7f28a41f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 >> 172.21.15.111:0/2134803913 conn(0x7f28a40b4640 msgr2=0x7f28a40ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 shutdown_connections 2023-12-04T05:39:53.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.404+0000 7f28abc23640 1 -- 172.21.15.111:0/2134803913 wait complete. 2023-12-04T05:39:53.416 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.518+0000 7f0cd7ae0640 1 Processor -- start 2023-12-04T05:39:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.518+0000 7f0cd7ae0640 1 -- start start 2023-12-04T05:39:53.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd7ae0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 0x7f0cd00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd7ae0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd00cfa10 con 0x7f0cd00b30e0 2023-12-04T05:39:53.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd7ae0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd01406f0 con 0x7f0cd0160e30 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd7ae0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00b4700 con 0x7f0cd00cf0f0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd607d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0cd00b30e0 0x7f0cd006a4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54842/0 (socket says 172.21.15.111:54842) 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd607d640 1 -- 172.21.15.111:0/3426596336 learned_addr learned my addr 172.21.15.111:0/3426596336 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd587c640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 0x7f0cd00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336611742 0 0) 0x7f0cd00cfa10 con 0x7f0cd00b30e0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cbc003520 con 0x7f0cd00b30e0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 71440242 0 0) 0x7f0cbc003520 con 0x7f0cd00b30e0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 msgr2=0x7f0cd00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 0x7f0cd00cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 >> v1:172.21.15.192:6789/0 conn(0x7f0cd0160e30 legacy=0x7f0cd0143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0161360 con 0x7f0cd00b30e0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f0cc80019f0 con 0x7f0cd00b30e0 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 >> v1:172.21.15.111:6789/0 conn(0x7f0cd00b30e0 legacy=0x7f0cd006a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cbc0039a0 0x7f0cbc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc004350 0x7f0cbc004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 0x7f0cbc005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00b4700 con 0x7f0cbc0039a0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc00b980 con 0x7f0cbc004350 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc00bb40 con 0x7f0cd00cf0f0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.519+0000 7f0cd587c640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cbc0039a0 0x7f0cbc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd607d640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 0x7f0cbc005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd507b640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc004350 0x7f0cbc004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd587c640 1 -- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 msgr2=0x7f0cbc005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd587c640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00cf0f0 0x7f0cbc005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd587c640 1 -- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc004350 msgr2=0x7f0cbc004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd587c640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc004350 0x7f0cbc004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd587c640 1 -- 172.21.15.111:0/3426596336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc003520 con 0x7f0cbc0039a0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd587c640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cbc0039a0 0x7f0cbc003d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ac0 tx=0x7f0cbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6393170e8d52c413 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.520+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc02c6b0 con 0x7f0cbc0039a0 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc02c810 con 0x7f0cbc0039a0 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd487a640 1 -- 172.21.15.111:0/3426596336 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc02cba0 con 0x7f0cbc0039a0 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/3426596336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cbc0039a0 msgr2=0x7f0cbc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/3426596336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cbc0039a0 0x7f0cbc003d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ac0 tx=0x7f0cbc0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd607d640 1 -- 172.21.15.111:0/3426596336 reap_dead start 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/3426596336 shutdown_connections 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/3426596336 >> 172.21.15.111:0/3426596336 conn(0x7f0cd00b1de0 msgr2=0x7f0cd00b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/3426596336 shutdown_connections 2023-12-04T05:39:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.521+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/3426596336 wait complete. 2023-12-04T05:39:53.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.522+0000 7f0cd7ae0640 1 Processor -- start 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.522+0000 7f0cd7ae0640 1 -- start start 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd7ae0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc0039a0 0x7f0cd0145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd7ae0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00ce950 0x7f0cd006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd7ae0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 0x7f0cd01f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd7ae0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd006fa40 con 0x7f0cd006f5c0 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd7ae0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01f3800 con 0x7f0cbc0039a0 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd7ae0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01f3520 con 0x7f0cd00ce950 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 0x7f0cd01f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 0x7f0cd01f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36266/0 (socket says 172.21.15.111:36266) 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 -- 172.21.15.111:0/2638794337 learned_addr learned my addr 172.21.15.111:0/2638794337 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd587c640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc0039a0 0x7f0cd0145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd507b640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00ce950 0x7f0cd006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 -- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00ce950 msgr2=0x7f0cd006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00ce950 0x7f0cd006f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 -- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc0039a0 msgr2=0x7f0cd0145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc0039a0 0x7f0cd0145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 -- 172.21.15.111:0/2638794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0144c40 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd507b640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00ce950 0x7f0cd006f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd587c640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc0039a0 0x7f0cd0145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.523+0000 7f0cd607d640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 0x7f0cd01f28e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0cd006c420 tx=0x7f0cc8007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.524+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800ea40 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.524+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc800eba0 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.524+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0144f20 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.524+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8013420 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.524+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd0146680 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.525+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd0145440 con 0x7f0cd006f5c0 2023-12-04T05:39:53.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.525+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0cc8013580 con 0x7f0cd006f5c0 2023-12-04T05:39:53.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.526+0000 7f0cc67fc640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0cb406b2a0 0x7f0cb406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.526+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cc8089fc0 con 0x7f0cd006f5c0 2023-12-04T05:39:53.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.526+0000 7f0cd587c640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0cb406b2a0 0x7f0cb406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.526+0000 7f0cd587c640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0cb406b2a0 0x7f0cb406d760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ac0 tx=0x7f0ccc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.530+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cd0145440 con 0x7f0cd006f5c0 2023-12-04T05:39:53.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.695+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0cd0145440 con 0x7f0cd006f5c0 2023-12-04T05:39:53.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.695+0000 7f0cc67fc640 1 -- 172.21.15.111:0/2638794337 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0cc8054e30 con 0x7f0cd006f5c0 2023-12-04T05:39:53.697 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0cb406b2a0 msgr2=0x7f0cb406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0cb406b2a0 0x7f0cb406d760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ac0 tx=0x7f0ccc030000 comp rx=0 tx=0).stop 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 msgr2=0x7f0cd01f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 0x7f0cd01f28e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0cd006c420 tx=0x7f0cc8007b10 comp rx=0 tx=0).stop 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 shutdown_connections 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0cb406b2a0 0x7f0cb406d760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cd006f5c0 0x7f0cd01f28e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0cd00ce950 0x7f0cd006f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 --2- 172.21.15.111:0/2638794337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cbc0039a0 0x7f0cd0145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 >> 172.21.15.111:0/2638794337 conn(0x7f0cd00b1de0 msgr2=0x7f0cd00b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 shutdown_connections 2023-12-04T05:39:53.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.697+0000 7f0cd7ae0640 1 -- 172.21.15.111:0/2638794337 wait complete. 2023-12-04T05:39:53.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:53.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.811+0000 7f10e0d97640 1 Processor -- start 2023-12-04T05:39:53.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.811+0000 7f10e0d97640 1 -- start start 2023-12-04T05:39:53.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.811+0000 7f10e0d97640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10dc158da0 0x7f10dc15c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10e0d97640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10dc15fbb0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10e0d97640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10dc15edf0 con 0x7f10dc06e4d0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10e0d97640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10dc15c6c0 con 0x7f10dc158da0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10dad9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f10dc06bbe0 0x7f10dc15c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54848/0 (socket says 172.21.15.111:54848) 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10dad9d640 1 -- 172.21.15.111:0/3084943510 learned_addr learned my addr 172.21.15.111:0/3084943510 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10da59c640 1 --2- 172.21.15.111:0/3084943510 >> v2:172.21.15.192:3300/0 conn(0x7f10dc158da0 0x7f10dc15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10d959a640 1 -- 172.21.15.111:0/3084943510 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688560755 0 0) 0x7f10dc15fbb0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10d959a640 1 -- 172.21.15.111:0/3084943510 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c4003520 con 0x7f10dc06bbe0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10da59c640 1 -- 172.21.15.111:0/3084943510 >> v1:172.21.15.192:6789/0 conn(0x7f10dc06e4d0 legacy=0x7f10dc06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10d959a640 1 -- 172.21.15.111:0/3084943510 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2679655547 0 0) 0x7f10c4003520 con 0x7f10dc06bbe0 2023-12-04T05:39:53.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10da59c640 1 -- 172.21.15.111:0/3084943510 >> v1:172.21.15.111:6789/0 conn(0x7f10dc06bbe0 legacy=0x7f10dc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.812+0000 7f10da59c640 1 -- 172.21.15.111:0/3084943510 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc15f0d0 con 0x7f10dc158da0 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.813+0000 7f10da59c640 1 --2- 172.21.15.111:0/3084943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc15c090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10d00097a0 tx=0x7f10d002b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df20a8be35f0d9d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.813+0000 7f10d959a640 1 -- 172.21.15.111:0/3084943510 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40019f0 con 0x7f10dc158da0 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.813+0000 7f10d959a640 1 -- 172.21.15.111:0/3084943510 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d002e050 con 0x7f10dc158da0 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.813+0000 7f10d959a640 1 -- 172.21.15.111:0/3084943510 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0030730 con 0x7f10dc158da0 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 -- 172.21.15.111:0/3084943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 msgr2=0x7f10dc15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 --2- 172.21.15.111:0/3084943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc15c090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10d00097a0 tx=0x7f10d002b680 comp rx=0 tx=0).stop 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 -- 172.21.15.111:0/3084943510 shutdown_connections 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 --2- 172.21.15.111:0/3084943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc15c090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 -- 172.21.15.111:0/3084943510 >> 172.21.15.111:0/3084943510 conn(0x7f10dc0b1b90 msgr2=0x7f10dc070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 -- 172.21.15.111:0/3084943510 shutdown_connections 2023-12-04T05:39:53.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.814+0000 7f10e0d97640 1 -- 172.21.15.111:0/3084943510 wait complete. 2023-12-04T05:39:53.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.815+0000 7f10e0d97640 1 Processor -- start 2023-12-04T05:39:53.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.815+0000 7f10e0d97640 1 -- start start 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10e0d97640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 0x7f10dc1f0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10e0d97640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10dc06e4d0 0x7f10dc0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10e0d97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10e0d97640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc15c0c0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10e0d97640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc0b2d60 con 0x7f10dc158da0 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10e0d97640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc154ea0 con 0x7f10dc06e4d0 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 0x7f10dc1f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 0x7f10dc1f0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36282/0 (socket says 172.21.15.111:36282) 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 -- 172.21.15.111:0/1430073726 learned_addr learned my addr 172.21.15.111:0/1430073726 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10dad9d640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10d9d9b640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10dc06e4d0 0x7f10dc0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 -- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10dc06e4d0 msgr2=0x7f10dc0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10dc06e4d0 0x7f10dc0b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 -- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 msgr2=0x7f10dc154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 -- 172.21.15.111:0/1430073726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009400 con 0x7f10dc06bbe0 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10da59c640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 0x7f10dc1f0320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10d0035040 tx=0x7f10d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.816+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0004220 con 0x7f10dc06bbe0 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.817+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d0004380 con 0x7f10dc06bbe0 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.817+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10dc15bde0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.817+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d003c8c0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.817+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc159620 con 0x7f10dc06bbe0 2023-12-04T05:39:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.818+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10dc069360 con 0x7f10dc06bbe0 2023-12-04T05:39:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.818+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f10d0039040 con 0x7f10dc06bbe0 2023-12-04T05:39:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.818+0000 7f10caffd640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10bc06b2a0 0x7f10bc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.818+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10d00ab4e0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.818+0000 7f10d9d9b640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10bc06b2a0 0x7f10bc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.819+0000 7f10d9d9b640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10bc06b2a0 0x7f10bc06d760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0718f0 tx=0x7f10cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.822+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10d0076500 con 0x7f10dc06bbe0 2023-12-04T05:39:53.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.987+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10dc1f2dd0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.988+0000 7f10caffd640 1 -- 172.21.15.111:0/1430073726 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10d00370a0 con 0x7f10dc06bbe0 2023-12-04T05:39:53.989 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10bc06b2a0 msgr2=0x7f10bc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10bc06b2a0 0x7f10bc06d760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0718f0 tx=0x7f10cc009040 comp rx=0 tx=0).stop 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 msgr2=0x7f10dc1f0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 0x7f10dc1f0320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10d0035040 tx=0x7f10d00046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 shutdown_connections 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10bc06b2a0 0x7f10bc06d760 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10dc158da0 0x7f10dc154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10dc06e4d0 0x7f10dc0b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 --2- 172.21.15.111:0/1430073726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10dc06bbe0 0x7f10dc1f0320 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:53.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.989+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 >> 172.21.15.111:0/1430073726 conn(0x7f10dc0b1b90 msgr2=0x7f10dc15e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:53.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.990+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 shutdown_connections 2023-12-04T05:39:53.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:53.990+0000 7f10e0d97640 1 -- 172.21.15.111:0/1430073726 wait complete. 2023-12-04T05:39:54.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:54.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.099+0000 7fede6e9b640 1 Processor -- start 2023-12-04T05:39:54.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.099+0000 7fede6e9b640 1 -- start start 2023-12-04T05:39:54.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7fede6e9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede0144430 0x7fede015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7fede6e9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fede00703f0 0x7fede0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7fede6e9b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede0070bd0 con 0x7fede00721b0 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7fede6e9b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede0144810 con 0x7fede00703f0 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7fede6e9b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede0070eb0 con 0x7fede0144430 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7feddffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fede00703f0 0x7fede0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7feddffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fede00703f0 0x7fede0071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36286/0 (socket says 172.21.15.111:36286) 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7feddffff640 1 -- 172.21.15.111:0/4162896596 learned_addr learned my addr 172.21.15.111:0/4162896596 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.100+0000 7fede4c37640 1 --2- 172.21.15.111:0/4162896596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede0144430 0x7fede015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddffff640 1 -- 172.21.15.111:0/4162896596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede0144430 msgr2=0x7fede015f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddffff640 1 --2- 172.21.15.111:0/4162896596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede0144430 0x7fede015f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddffff640 1 -- 172.21.15.111:0/4162896596 >> v1:172.21.15.192:6789/0 conn(0x7fede00721b0 legacy=0x7fede006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddffff640 1 -- 172.21.15.111:0/4162896596 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0071190 con 0x7fede00703f0 2023-12-04T05:39:54.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddffff640 1 --2- 172.21.15.111:0/4162896596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00703f0 0x7fede0071c70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009870 tx=0x7fedd402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5fef726145291b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddf7fe640 1 -- 172.21.15.111:0/4162896596 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4030840 con 0x7fede00703f0 2023-12-04T05:39:54.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddf7fe640 1 -- 172.21.15.111:0/4162896596 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd40309a0 con 0x7fede00703f0 2023-12-04T05:39:54.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.101+0000 7feddf7fe640 1 -- 172.21.15.111:0/4162896596 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4030d30 con 0x7fede00703f0 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 -- 172.21.15.111:0/4162896596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00703f0 msgr2=0x7fede0071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 --2- 172.21.15.111:0/4162896596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00703f0 0x7fede0071c70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009870 tx=0x7fedd402b980 comp rx=0 tx=0).stop 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 -- 172.21.15.111:0/4162896596 shutdown_connections 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 --2- 172.21.15.111:0/4162896596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00703f0 0x7fede0071c70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 --2- 172.21.15.111:0/4162896596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede0144430 0x7fede015f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 -- 172.21.15.111:0/4162896596 >> 172.21.15.111:0/4162896596 conn(0x7fede006a4c0 msgr2=0x7fede006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 -- 172.21.15.111:0/4162896596 shutdown_connections 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.102+0000 7fede6e9b640 1 -- 172.21.15.111:0/4162896596 wait complete. 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.103+0000 7fede6e9b640 1 Processor -- start 2023-12-04T05:39:54.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.103+0000 7fede6e9b640 1 -- start start 2023-12-04T05:39:54.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede6e9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede00703f0 0x7fede013f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede6e9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 0x7fede0139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede6e9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede0144430 0x7fede01389e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede6e9b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01396d0 con 0x7fede00721b0 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede6e9b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0139950 con 0x7fede0144430 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede6e9b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede0139c30 con 0x7fede00703f0 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 0x7fede0139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 0x7fede0139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36300/0 (socket says 172.21.15.111:36300) 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 -- 172.21.15.111:0/898445103 learned_addr learned my addr 172.21.15.111:0/898445103 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede4c37640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede00703f0 0x7fede013f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede5438640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede0144430 0x7fede01389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 -- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede00703f0 msgr2=0x7fede013f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede00703f0 0x7fede013f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 -- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede0144430 msgr2=0x7fede01389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede0144430 0x7fede01389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7feddffff640 1 -- 172.21.15.111:0/898445103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd40094d0 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede4c37640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede00703f0 0x7fede013f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.104+0000 7fede5438640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede0144430 0x7fede01389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.105+0000 7feddffff640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 0x7fede0139190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000c00 tx=0x7fedd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.105+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4039c90 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.105+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4039df0 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.105+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4043640 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.105+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede0139f10 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.105+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede013a4d0 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.106+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede00693d0 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.106+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fedd40437a0 con 0x7fede00721b0 2023-12-04T05:39:54.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.107+0000 7feddd7fa640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fedcc06b2d0 0x7fedcc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.107+0000 7fede4c37640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fedcc06b2d0 0x7fedcc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.107+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedd403e070 con 0x7fede00721b0 2023-12-04T05:39:54.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.107+0000 7fede4c37640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fedcc06b2d0 0x7fedcc06d790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009a20 tx=0x7fedd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.111+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fedd4077a60 con 0x7fede00721b0 2023-12-04T05:39:54.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.289+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fede006d100 con 0x7fede00721b0 2023-12-04T05:39:54.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.289+0000 7feddd7fa640 1 -- 172.21.15.111:0/898445103 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fedd407b080 con 0x7fede00721b0 2023-12-04T05:39:54.291 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fedcc06b2d0 msgr2=0x7fedcc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fedcc06b2d0 0x7fedcc06d790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009a20 tx=0x7fedd0008040 comp rx=0 tx=0).stop 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 msgr2=0x7fede0139190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 0x7fede0139190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000c00 tx=0x7fedd4004730 comp rx=0 tx=0).stop 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 shutdown_connections 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fedcc06b2d0 0x7fedcc06d790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede0144430 0x7fede01389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fede00721b0 0x7fede0139190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 --2- 172.21.15.111:0/898445103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fede00703f0 0x7fede013f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 >> 172.21.15.111:0/898445103 conn(0x7fede006a4c0 msgr2=0x7fede0143760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 shutdown_connections 2023-12-04T05:39:54.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.291+0000 7fede6e9b640 1 -- 172.21.15.111:0/898445103 wait complete. 2023-12-04T05:39:54.304 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-12-04T05:39:54.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:54.305 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-12-04T05:39:54.331 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:39:54.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:39:54.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:54.332 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-04T05:39:54.387 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:39:54.387 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:54.406 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:39:54.406 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:54.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:54.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.563+0000 7f85042ad640 1 Processor -- start 2023-12-04T05:39:54.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.563+0000 7f85042ad640 1 -- start start 2023-12-04T05:39:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f85042ad640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84fc13cee0 0x7f84fc06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f85042ad640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84fc15aba0 0x7f84fc06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f85042ad640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc0aea20 con 0x7f84fc140200 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f85042ad640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc13d2c0 con 0x7f84fc15aba0 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f85042ad640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc15b3a0 con 0x7f84fc13cee0 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84fc15aba0 0x7f84fc06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84fc15aba0 0x7f84fc06c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36316/0 (socket says 172.21.15.111:36316) 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 -- 172.21.15.111:0/3110732481 learned_addr learned my addr 172.21.15.111:0/3110732481 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f8502049640 1 --2- 172.21.15.111:0/3110732481 >> v2:172.21.15.192:3300/0 conn(0x7f84fc13cee0 0x7f84fc06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 -- 172.21.15.111:0/3110732481 >> v2:172.21.15.192:3300/0 conn(0x7f84fc13cee0 msgr2=0x7f84fc06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 --2- 172.21.15.111:0/3110732481 >> v2:172.21.15.192:3300/0 conn(0x7f84fc13cee0 0x7f84fc06c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 -- 172.21.15.111:0/3110732481 >> v1:172.21.15.192:6789/0 conn(0x7f84fc140200 legacy=0x7f84fc15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f850284a640 1 -- 172.21.15.111:0/3110732481 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc15b0c0 con 0x7f84fc15aba0 2023-12-04T05:39:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.564+0000 7f8502049640 1 --2- 172.21.15.111:0/3110732481 >> v2:172.21.15.192:3300/0 conn(0x7f84fc13cee0 0x7f84fc06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.565+0000 7f850284a640 1 --2- 172.21.15.111:0/3110732481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc15aba0 0x7f84fc06c790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0b2280 tx=0x7f84f402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77708f149fc29c2f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.565+0000 7f8501047640 1 -- 172.21.15.111:0/3110732481 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f402c9f0 con 0x7f84fc15aba0 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.565+0000 7f8501047640 1 -- 172.21.15.111:0/3110732481 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f402cb50 con 0x7f84fc15aba0 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.565+0000 7f8501047640 1 -- 172.21.15.111:0/3110732481 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40343f0 con 0x7f84fc15aba0 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.565+0000 7f85042ad640 1 -- 172.21.15.111:0/3110732481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc15aba0 msgr2=0x7f84fc06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.565+0000 7f85042ad640 1 --2- 172.21.15.111:0/3110732481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc15aba0 0x7f84fc06c790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0b2280 tx=0x7f84f402be50 comp rx=0 tx=0).stop 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.566+0000 7f85042ad640 1 -- 172.21.15.111:0/3110732481 shutdown_connections 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.566+0000 7f85042ad640 1 --2- 172.21.15.111:0/3110732481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc15aba0 0x7f84fc06c790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.566+0000 7f85042ad640 1 --2- 172.21.15.111:0/3110732481 >> v2:172.21.15.192:3300/0 conn(0x7f84fc13cee0 0x7f84fc06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.566+0000 7f85042ad640 1 -- 172.21.15.111:0/3110732481 >> 172.21.15.111:0/3110732481 conn(0x7f84fc06a4c0 msgr2=0x7f84fc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.566+0000 7f85042ad640 1 -- 172.21.15.111:0/3110732481 shutdown_connections 2023-12-04T05:39:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.566+0000 7f85042ad640 1 -- 172.21.15.111:0/3110732481 wait complete. 2023-12-04T05:39:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 Processor -- start 2023-12-04T05:39:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 -- start start 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84fc13cee0 0x7f84fc13ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 0x7f84fc0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84fc15aba0 0x7f84fc147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc13d870 con 0x7f84fc140200 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1478a0 con 0x7f84fc15aba0 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f85042ad640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc147b80 con 0x7f84fc13cee0 2023-12-04T05:39:54.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.567+0000 7f8501848640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 0x7f84fc0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 0x7f84fc0b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36328/0 (socket says 172.21.15.111:36328) 2023-12-04T05:39:54.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 -- 172.21.15.111:0/3664460486 learned_addr learned my addr 172.21.15.111:0/3664460486 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:54.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8502049640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84fc13cee0 0x7f84fc13ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f850284a640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84fc15aba0 0x7f84fc147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 -- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84fc13cee0 msgr2=0x7f84fc13ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84fc13cee0 0x7f84fc13ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 -- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84fc15aba0 msgr2=0x7f84fc147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84fc15aba0 0x7f84fc147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 -- 172.21.15.111:0/3664460486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4009b30 con 0x7f84fc140200 2023-12-04T05:39:54.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f850284a640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84fc15aba0 0x7f84fc147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f8501848640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 0x7f84fc0b2f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c430 tx=0x7f84ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011790 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec0118f0 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011ce0 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc147e60 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.568+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc147570 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.569+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc0703d0 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.570+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f84ec01c460 con 0x7f84fc140200 2023-12-04T05:39:54.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.570+0000 7f84f2ffd640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84e006b2a0 0x7f84e006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.570+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84ec089c10 con 0x7f84fc140200 2023-12-04T05:39:54.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.570+0000 7f8502049640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84e006b2a0 0x7f84e006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.570+0000 7f8502049640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84e006b2a0 0x7f84e006d760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84f8008850 tx=0x7f84f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.574+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84fc0703d0 con 0x7f84fc140200 2023-12-04T05:39:54.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.740+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84fc0703d0 con 0x7f84fc140200 2023-12-04T05:39:54.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.742+0000 7f84f2ffd640 1 -- 172.21.15.111:0/3664460486 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f84fc0703d0 con 0x7f84fc140200 2023-12-04T05:39:54.743 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:54.743 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.743+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84e006b2a0 msgr2=0x7f84e006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.743+0000 7f85042ad640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84e006b2a0 0x7f84e006d760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84f8008850 tx=0x7f84f8008040 comp rx=0 tx=0).stop 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.743+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 msgr2=0x7f84fc0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.743+0000 7f85042ad640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 0x7f84fc0b2f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00c430 tx=0x7f84ec00c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 shutdown_connections 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f84e006b2a0 0x7f84e006d760 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84fc15aba0 0x7f84fc147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc140200 0x7f84fc0b2f60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 --2- 172.21.15.111:0/3664460486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84fc13cee0 0x7f84fc13ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 >> 172.21.15.111:0/3664460486 conn(0x7f84fc06a4c0 msgr2=0x7f84fc15c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:54.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 shutdown_connections 2023-12-04T05:39:54.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.744+0000 7f85042ad640 1 -- 172.21.15.111:0/3664460486 wait complete. 2023-12-04T05:39:54.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:54.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:54.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.858+0000 7f2625189640 1 Processor -- start 2023-12-04T05:39:54.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.858+0000 7f2625189640 1 -- start start 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f2625189640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 0x7f26200cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f2625189640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 0x7f2620160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f2625189640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262015ddc0 con 0x7f262015d480 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f2625189640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26200d04a0 con 0x7f26200d0010 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f2625189640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262015e0a0 con 0x7f262015d860 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f261e59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 0x7f26200cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f261f59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 0x7f2620160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261e59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 0x7f26200cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:37042/0 (socket says 172.21.15.111:37042) 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.859+0000 7f261ed9d640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f262015d480 0x7f26200d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50840/0 (socket says 172.21.15.111:50840) 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261e59c640 1 -- 172.21.15.111:0/3666969542 learned_addr learned my addr 172.21.15.111:0/3666969542 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261f59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 0x7f2620160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46778/0 (socket says 172.21.15.111:46778) 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261dd9b640 1 -- 172.21.15.111:0/3666969542 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690986355 0 0) 0x7f262015ddc0 con 0x7f262015d480 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261dd9b640 1 -- 172.21.15.111:0/3666969542 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2608003520 con 0x7f262015d480 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261e59c640 1 -- 172.21.15.111:0/3666969542 >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 msgr2=0x7f2620160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261e59c640 1 --2- 172.21.15.111:0/3666969542 >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 0x7f2620160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261e59c640 1 -- 172.21.15.111:0/3666969542 >> v1:172.21.15.192:6789/0 conn(0x7f262015d480 legacy=0x7f26200d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261e59c640 1 -- 172.21.15.111:0/3666969542 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262015e380 con 0x7f262015d860 2023-12-04T05:39:54.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.860+0000 7f261f59e640 1 --2- 172.21.15.111:0/3666969542 >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 0x7f2620160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:54.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f261e59c640 1 --2- 172.21.15.111:0/3666969542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 0x7f26200cfad0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f261402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e49a2938f7b2bea server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f261dd9b640 1 -- 172.21.15.111:0/3666969542 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614030840 con 0x7f262015d860 2023-12-04T05:39:54.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f261dd9b640 1 -- 172.21.15.111:0/3666969542 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26140309a0 con 0x7f262015d860 2023-12-04T05:39:54.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f261dd9b640 1 -- 172.21.15.111:0/3666969542 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614030d10 con 0x7f262015d860 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 -- 172.21.15.111:0/3666969542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 msgr2=0x7f26200cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 --2- 172.21.15.111:0/3666969542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 0x7f26200cfad0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f261402b980 comp rx=0 tx=0).stop 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 -- 172.21.15.111:0/3666969542 shutdown_connections 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 --2- 172.21.15.111:0/3666969542 >> v2:172.21.15.192:3300/0 conn(0x7f26200d0010 0x7f2620160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 --2- 172.21.15.111:0/3666969542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d860 0x7f26200cfad0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 -- 172.21.15.111:0/3666969542 >> 172.21.15.111:0/3666969542 conn(0x7f262009c540 msgr2=0x7f26200d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 -- 172.21.15.111:0/3666969542 shutdown_connections 2023-12-04T05:39:54.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.861+0000 7f2625189640 1 -- 172.21.15.111:0/3666969542 wait complete. 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.863+0000 7f2625189640 1 Processor -- start 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.863+0000 7f2625189640 1 -- start start 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 0x7f26200cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d480 0x7f2620159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f262015d860 0x7f2620159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26200cc1d0 con 0x7f26200d0010 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26200cb320 con 0x7f262015d860 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26200cb5c0 con 0x7f262015d480 2023-12-04T05:39:54.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 0x7f26200cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 0x7f26200cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36334/0 (socket says 172.21.15.111:36334) 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 -- 172.21.15.111:0/1514980619 learned_addr learned my addr 172.21.15.111:0/1514980619 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261e59c640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d480 0x7f2620159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261f59e640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f262015d860 0x7f2620159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 -- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d480 msgr2=0x7f2620159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d480 0x7f2620159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 -- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f262015d860 msgr2=0x7f2620159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f262015d860 0x7f2620159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 -- 172.21.15.111:0/1514980619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2618009400 con 0x7f26200d0010 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f261ed9d640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 0x7f26200cc660 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f261000c840 tx=0x7f261000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000ce30 con 0x7f26200d0010 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261000a640 con 0x7f26200d0010 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000a9d0 con 0x7f26200d0010 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.864+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26140094d0 con 0x7f26200d0010 2023-12-04T05:39:54.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.865+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26200cbb80 con 0x7f26200d0010 2023-12-04T05:39:54.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.865+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2620067980 con 0x7f26200d0010 2023-12-04T05:39:54.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.866+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f261000ab30 con 0x7f26200d0010 2023-12-04T05:39:54.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.866+0000 7f26077fe640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f260c06b2a0 0x7f260c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:54.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.866+0000 7f261e59c640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f260c06b2a0 0x7f260c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:54.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.866+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26100892b0 con 0x7f26200d0010 2023-12-04T05:39:54.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.867+0000 7f261e59c640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f260c06b2a0 0x7f260c06d760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f261403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:54.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:54.871+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2610054260 con 0x7f26200d0010 2023-12-04T05:39:55.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.037+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2620069360 con 0x7f26200d0010 2023-12-04T05:39:55.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.037+0000 7f26077fe640 1 -- 172.21.15.111:0/1514980619 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2610057880 con 0x7f26200d0010 2023-12-04T05:39:55.039 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f260c06b2a0 msgr2=0x7f260c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f260c06b2a0 0x7f260c06d760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f261403b040 comp rx=0 tx=0).stop 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 msgr2=0x7f26200cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 0x7f26200cc660 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f261000c840 tx=0x7f261000cc40 comp rx=0 tx=0).stop 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 shutdown_connections 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f260c06b2a0 0x7f260c06d760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f262015d860 0x7f2620159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f262015d480 0x7f2620159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 --2- 172.21.15.111:0/1514980619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26200d0010 0x7f26200cc660 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 >> 172.21.15.111:0/1514980619 conn(0x7f262009c540 msgr2=0x7f262006cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 shutdown_connections 2023-12-04T05:39:55.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.039+0000 7f2625189640 1 -- 172.21.15.111:0/1514980619 wait complete. 2023-12-04T05:39:55.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:55.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.152+0000 7f5700733640 1 Processor -- start 2023-12-04T05:39:55.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.152+0000 7f5700733640 1 -- start start 2023-12-04T05:39:55.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f5700733640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 0x7f56f80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f5700733640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80cfa10 con 0x7f56f80b30e0 2023-12-04T05:39:55.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f5700733640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f81406f0 con 0x7f56f8160e30 2023-12-04T05:39:55.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f5700733640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80b4700 con 0x7f56f80cf0f0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f56fecd0640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f56f80b30e0 0x7f56f806a4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54852/0 (socket says 172.21.15.111:54852) 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f56fecd0640 1 -- 172.21.15.111:0/2878345489 learned_addr learned my addr 172.21.15.111:0/2878345489 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.153+0000 7f56fe4cf640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 0x7f56f80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495393149 0 0) 0x7f56f80cfa10 con 0x7f56f80b30e0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e4003520 con 0x7f56f80b30e0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210668670 0 0) 0x7f56f81406f0 con 0x7f56f8160e30 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f80cfa10 con 0x7f56f8160e30 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173702969 0 0) 0x7f56e4003520 con 0x7f56f80b30e0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 msgr2=0x7f56f80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 0x7f56f80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 >> v1:172.21.15.192:6789/0 conn(0x7f56f8160e30 legacy=0x7f56f8143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8161360 con 0x7f56f80b30e0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f56f0001670 con 0x7f56f80b30e0 2023-12-04T05:39:55.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 >> v1:172.21.15.111:6789/0 conn(0x7f56f80b30e0 legacy=0x7f56f806a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 0x7f56e4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e4004d00 0x7f56e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80b4700 con 0x7f56e4004090 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e400ba60 con 0x7f56e4004d00 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e400bc20 con 0x7f56f80cf0f0 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fe4cf640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 0x7f56e4003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fecd0640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e4004d00 0x7f56e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fdcce640 1 -- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 msgr2=0x7f56e4003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80cf0f0 0x7f56e4003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fdcce640 1 -- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e4004d00 msgr2=0x7f56e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e4004d00 0x7f56e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.154+0000 7f56fdcce640 1 -- 172.21.15.111:0/2878345489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4003520 con 0x7f56e4004090 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.155+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56e40047c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f56e80018a0 tx=0x7f56e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ae610c3814056 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.155+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80052d0 con 0x7f56e4004090 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.155+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8005430 con 0x7f56e4004090 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.155+0000 7f56fd4cd640 1 -- 172.21.15.111:0/2878345489 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8003810 con 0x7f56e4004090 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f5700733640 1 -- 172.21.15.111:0/2878345489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 msgr2=0x7f56e40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f5700733640 1 --2- 172.21.15.111:0/2878345489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56e40047c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f56e80018a0 tx=0x7f56e40057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f56fecd0640 1 -- 172.21.15.111:0/2878345489 reap_dead start 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f5700733640 1 -- 172.21.15.111:0/2878345489 shutdown_connections 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f5700733640 1 -- 172.21.15.111:0/2878345489 >> 172.21.15.111:0/2878345489 conn(0x7f56f80b1de0 msgr2=0x7f56f80b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f5700733640 1 -- 172.21.15.111:0/2878345489 shutdown_connections 2023-12-04T05:39:55.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.156+0000 7f5700733640 1 -- 172.21.15.111:0/2878345489 wait complete. 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.157+0000 7f5700733640 1 Processor -- start 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.157+0000 7f5700733640 1 -- start start 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56f815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56f8142f10 0x7f56f8143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80c5050 0x7f56f80c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8141d70 con 0x7f56e4004090 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8142190 con 0x7f56f8142f10 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8142430 con 0x7f56f80c5050 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56f815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56f815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36352/0 (socket says 172.21.15.111:36352) 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 -- 172.21.15.111:0/2299597507 learned_addr learned my addr 172.21.15.111:0/2299597507 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56f8142f10 0x7f56f8143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fecd0640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80c5050 0x7f56f80c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 -- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80c5050 msgr2=0x7f56f80c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80c5050 0x7f56f80c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 -- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56f8142f10 msgr2=0x7f56f8143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56f8142f10 0x7f56f8143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 -- 172.21.15.111:0/2299597507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8142710 con 0x7f56e4004090 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56fe4cf640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56f815b910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56f4011e30 tx=0x7f56f400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f400e040 con 0x7f56e4004090 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f400f040 con 0x7f56e4004090 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f81429f0 con 0x7f56e4004090 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4009c00 con 0x7f56e4004090 2023-12-04T05:39:55.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.158+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f815be50 con 0x7f56e4004090 2023-12-04T05:39:55.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.159+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f806f240 con 0x7f56e4004090 2023-12-04T05:39:55.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.160+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f56f4015850 con 0x7f56e4004090 2023-12-04T05:39:55.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.160+0000 7f56eeffd640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f56dc06b2a0 0x7f56dc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.160+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56f408a880 con 0x7f56e4004090 2023-12-04T05:39:55.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.160+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f56dc06b2a0 0x7f56dc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.161+0000 7f56fdcce640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f56dc06b2a0 0x7f56dc06d760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f56e8034000 tx=0x7f56e8031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.164+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56f4055780 con 0x7f56e4004090 2023-12-04T05:39:55.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.332+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56f80b44f0 con 0x7f56e4004090 2023-12-04T05:39:55.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.332+0000 7f56eeffd640 1 -- 172.21.15.111:0/2299597507 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56f4058da0 con 0x7f56e4004090 2023-12-04T05:39:55.334 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:55.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f56dc06b2a0 msgr2=0x7f56dc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f56dc06b2a0 0x7f56dc06d760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f56e8034000 tx=0x7f56e8031000 comp rx=0 tx=0).stop 2023-12-04T05:39:55.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 msgr2=0x7f56f815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56f815b910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56f4011e30 tx=0x7f56f400f580 comp rx=0 tx=0).stop 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 shutdown_connections 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f56dc06b2a0 0x7f56dc06d760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56f80c5050 0x7f56f80c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56f8142f10 0x7f56f8143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 --2- 172.21.15.111:0/2299597507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56e4004090 0x7f56f815b910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 >> 172.21.15.111:0/2299597507 conn(0x7f56f80b1de0 msgr2=0x7f56f806f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 shutdown_connections 2023-12-04T05:39:55.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.335+0000 7f5700733640 1 -- 172.21.15.111:0/2299597507 wait complete. 2023-12-04T05:39:55.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:55.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.448+0000 7f435929b640 1 Processor -- start 2023-12-04T05:39:55.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.448+0000 7f435929b640 1 -- start start 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.449+0000 7f435929b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f435415e840 0x7f435415f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.449+0000 7f435929b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43540b41b0 0x7f4354160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.449+0000 7f435929b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435415ec20 con 0x7f43540d17b0 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.449+0000 7f435929b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435415f8a0 con 0x7f435415e840 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.449+0000 7f435929b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435415fb80 con 0x7f43540b41b0 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f435415e840 0x7f435415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f43527fc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43540b41b0 0x7f4354160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f435415e840 0x7f435415f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46830/0 (socket says 172.21.15.111:46830) 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 -- 172.21.15.111:0/265816873 learned_addr learned my addr 172.21.15.111:0/265816873 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f43537fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f43540d17b0 0x7f43540cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50848/0 (socket says 172.21.15.111:50848) 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 -- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43540b41b0 msgr2=0x7f4354160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 --2- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43540b41b0 0x7f4354160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 -- 172.21.15.111:0/265816873 >> v1:172.21.15.192:6789/0 conn(0x7f43540d17b0 legacy=0x7f43540cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.450+0000 7f4352ffd640 1 -- 172.21.15.111:0/265816873 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435415fe60 con 0x7f435415e840 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f4352ffd640 1 --2- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f435415e840 0x7f435415f070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4344009e50 tx=0x7f434402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a656f0f2127033c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f4351ffb640 1 -- 172.21.15.111:0/265816873 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434402cb00 con 0x7f435415e840 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f4351ffb640 1 -- 172.21.15.111:0/265816873 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434402cc60 con 0x7f435415e840 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f4351ffb640 1 -- 172.21.15.111:0/265816873 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440343f0 con 0x7f435415e840 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 -- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f435415e840 msgr2=0x7f435415f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 --2- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f435415e840 0x7f435415f070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4344009e50 tx=0x7f434402bf60 comp rx=0 tx=0).stop 2023-12-04T05:39:55.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 -- 172.21.15.111:0/265816873 shutdown_connections 2023-12-04T05:39:55.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 --2- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43540b41b0 0x7f4354160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 --2- 172.21.15.111:0/265816873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f435415e840 0x7f435415f070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 -- 172.21.15.111:0/265816873 >> 172.21.15.111:0/265816873 conn(0x7f435409c4c0 msgr2=0x7f43540d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 -- 172.21.15.111:0/265816873 shutdown_connections 2023-12-04T05:39:55.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.451+0000 7f435929b640 1 -- 172.21.15.111:0/265816873 wait complete. 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 Processor -- start 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 -- start start 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 0x7f43541539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43540d17b0 0x7f43541533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f435415e840 0x7f435415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540ca630 con 0x7f43540b41b0 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435415cdc0 con 0x7f43540d17b0 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f435929b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c6dc0 con 0x7f435415e840 2023-12-04T05:39:55.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f4352ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 0x7f43541539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f4352ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 0x7f43541539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36368/0 (socket says 172.21.15.111:36368) 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.453+0000 7f4352ffd640 1 -- 172.21.15.111:0/3128519864 learned_addr learned my addr 172.21.15.111:0/3128519864 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f43527fc640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43540d17b0 0x7f43541533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f43537fe640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f435415e840 0x7f435415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f4352ffd640 1 -- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f435415e840 msgr2=0x7f435415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f4352ffd640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f435415e840 0x7f435415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f4352ffd640 1 -- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43540d17b0 msgr2=0x7f43541533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f4352ffd640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43540d17b0 0x7f43541533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f4352ffd640 1 -- 172.21.15.111:0/3128519864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344009b30 con 0x7f43540b41b0 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f43527fc640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43540d17b0 0x7f43541533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f4352ffd640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 0x7f43541539f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4344036040 tx=0x7f4344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344009510 con 0x7f43540b41b0 2023-12-04T05:39:55.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434403e450 con 0x7f43540b41b0 2023-12-04T05:39:55.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43540c70a0 con 0x7f43540b41b0 2023-12-04T05:39:55.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434403e800 con 0x7f43540b41b0 2023-12-04T05:39:55.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.454+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43540ca2c0 con 0x7f43540b41b0 2023-12-04T05:39:55.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.455+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43540c75c0 con 0x7f43540b41b0 2023-12-04T05:39:55.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.456+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f434403e960 con 0x7f43540b41b0 2023-12-04T05:39:55.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.456+0000 7f433b7fe640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f434006b2a0 0x7f434006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.456+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43440ac8d0 con 0x7f43540b41b0 2023-12-04T05:39:55.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.456+0000 7f43527fc640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f434006b2a0 0x7f434006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.456+0000 7f43527fc640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f434006b2a0 0x7f434006d760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f43540b8c60 tx=0x7f4348008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.460+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43540c75c0 con 0x7f43540b41b0 2023-12-04T05:39:55.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.629+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43540c75c0 con 0x7f43540b41b0 2023-12-04T05:39:55.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.629+0000 7f433b7fe640 1 -- 172.21.15.111:0/3128519864 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43540c75c0 con 0x7f43540b41b0 2023-12-04T05:39:55.630 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f434006b2a0 msgr2=0x7f434006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f434006b2a0 0x7f434006d760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f43540b8c60 tx=0x7f4348008040 comp rx=0 tx=0).stop 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 msgr2=0x7f43541539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 0x7f43541539f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4344036040 tx=0x7f4344004a40 comp rx=0 tx=0).stop 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 shutdown_connections 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f434006b2a0 0x7f434006d760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f435415e840 0x7f435415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43540d17b0 0x7f43541533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 --2- 172.21.15.111:0/3128519864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43540b41b0 0x7f43541539f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 >> 172.21.15.111:0/3128519864 conn(0x7f435409c4c0 msgr2=0x7f43540d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 shutdown_connections 2023-12-04T05:39:55.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.631+0000 7f435929b640 1 -- 172.21.15.111:0/3128519864 wait complete. 2023-12-04T05:39:55.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:55.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.743+0000 7f630ce07640 1 Processor -- start 2023-12-04T05:39:55.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.743+0000 7f630ce07640 1 -- start start 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f630ce07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f630ce07640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6308140200 0x7f630806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f630ce07640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f630815a490 0x7f630815a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f630ce07640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63080aea20 con 0x7f6308140200 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f630ce07640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630813d2c0 con 0x7f630815a490 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f630ce07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630815b3a0 con 0x7f630813cee0 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f6305d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6308140200 0x7f630806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f6305d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6308140200 0x7f630806c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36376/0 (socket says 172.21.15.111:36376) 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.744+0000 7f6305d9b640 1 -- 172.21.15.111:0/3873763953 learned_addr learned my addr 172.21.15.111:0/3873763953 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6306d9d640 1 --2- 172.21.15.111:0/3873763953 >> v2:172.21.15.192:3300/0 conn(0x7f630815a490 0x7f630815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f630659c640 1 --2- 172.21.15.111:0/3873763953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6305d9b640 1 -- 172.21.15.111:0/3873763953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 msgr2=0x7f630806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6305d9b640 1 --2- 172.21.15.111:0/3873763953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630806c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6305d9b640 1 -- 172.21.15.111:0/3873763953 >> v2:172.21.15.192:3300/0 conn(0x7f630815a490 msgr2=0x7f630815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6305d9b640 1 --2- 172.21.15.111:0/3873763953 >> v2:172.21.15.192:3300/0 conn(0x7f630815a490 0x7f630815a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6305d9b640 1 -- 172.21.15.111:0/3873763953 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630815ae60 con 0x7f6308140200 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.745+0000 7f6305d9b640 1 --2- 172.21.15.111:0/3873763953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f630806c790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f63080b2230 tx=0x7f62fc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b3d3df196a247f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630559a640 1 -- 172.21.15.111:0/3873763953 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0317d0 con 0x7f6308140200 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630559a640 1 -- 172.21.15.111:0/3873763953 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc031930 con 0x7f6308140200 2023-12-04T05:39:55.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630559a640 1 -- 172.21.15.111:0/3873763953 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc031cc0 con 0x7f6308140200 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 -- 172.21.15.111:0/3873763953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 msgr2=0x7f630806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 --2- 172.21.15.111:0/3873763953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f630806c790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f63080b2230 tx=0x7f62fc02b870 comp rx=0 tx=0).stop 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 -- 172.21.15.111:0/3873763953 shutdown_connections 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 --2- 172.21.15.111:0/3873763953 >> v2:172.21.15.192:3300/0 conn(0x7f630815a490 0x7f630815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 --2- 172.21.15.111:0/3873763953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f630806c790 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 --2- 172.21.15.111:0/3873763953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630806c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 -- 172.21.15.111:0/3873763953 >> 172.21.15.111:0/3873763953 conn(0x7f630806a4c0 msgr2=0x7f630806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 -- 172.21.15.111:0/3873763953 shutdown_connections 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.746+0000 7f630ce07640 1 -- 172.21.15.111:0/3873763953 wait complete. 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 Processor -- start 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 -- start start 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f6308137db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630815a490 0x7f630813e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081f4070 con 0x7f6308140200 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081f3460 con 0x7f630815a490 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f630ce07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63081f3740 con 0x7f630813cee0 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f6305d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f6308137db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f6305d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f6308137db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36384/0 (socket says 172.21.15.111:36384) 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.748+0000 7f6305d9b640 1 -- 172.21.15.111:0/1603755149 learned_addr learned my addr 172.21.15.111:0/1603755149 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6306d9d640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630815a490 0x7f630813e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f630659c640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6305d9b640 1 -- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 msgr2=0x7f630814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6305d9b640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630814fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6305d9b640 1 -- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630815a490 msgr2=0x7f630813e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6305d9b640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630815a490 0x7f630813e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6305d9b640 1 -- 172.21.15.111:0/1603755149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0094d0 con 0x7f6308140200 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f6305d9b640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f6308137db0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f62fc02b8b0 tx=0x7f62fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc032870 con 0x7f6308140200 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0329d0 con 0x7f6308140200 2023-12-04T05:39:55.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63081f3a20 con 0x7f6308140200 2023-12-04T05:39:55.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc032d80 con 0x7f6308140200 2023-12-04T05:39:55.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.749+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6308150400 con 0x7f6308140200 2023-12-04T05:39:55.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.750+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63081525f0 con 0x7f6308140200 2023-12-04T05:39:55.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.750+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f62fc030070 con 0x7f6308140200 2023-12-04T05:39:55.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.751+0000 7f62f6ffd640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f62e806b2d0 0x7f62e806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:55.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.751+0000 7f630659c640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f62e806b2d0 0x7f62e806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:55.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.751+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62fc0aba60 con 0x7f6308140200 2023-12-04T05:39:55.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.751+0000 7f630659c640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f62e806b2d0 0x7f62e806d790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62f8006fd0 tx=0x7f62f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:55.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.755+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62fc076a10 con 0x7f6308140200 2023-12-04T05:39:55.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.932+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f63080693d0 con 0x7f6308140200 2023-12-04T05:39:55.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.932+0000 7f62f6ffd640 1 -- 172.21.15.111:0/1603755149 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f62fc07a030 con 0x7f6308140200 2023-12-04T05:39:55.934 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f62e806b2d0 msgr2=0x7f62e806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f62e806b2d0 0x7f62e806d790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62f8006fd0 tx=0x7f62f8008040 comp rx=0 tx=0).stop 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 msgr2=0x7f6308137db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f6308137db0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f62fc02b8b0 tx=0x7f62fc0046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 shutdown_connections 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f62e806b2d0 0x7f62e806d790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630815a490 0x7f630813e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6308140200 0x7f6308137db0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 --2- 172.21.15.111:0/1603755149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630813cee0 0x7f630814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 >> 172.21.15.111:0/1603755149 conn(0x7f630806a4c0 msgr2=0x7f63080cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:55.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 shutdown_connections 2023-12-04T05:39:55.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:55.934+0000 7f630ce07640 1 -- 172.21.15.111:0/1603755149 wait complete. 2023-12-04T05:39:55.947 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:39:55.947 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:39:55.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.064+0000 7fdb33eb4640 1 Processor -- start 2023-12-04T05:39:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.064+0000 7fdb33eb4640 1 -- start start 2023-12-04T05:39:56.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.065+0000 7fdb33eb4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 0x7fdb2c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.065+0000 7fdb33eb4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2c06f100 con 0x7fdb2c0cfaf0 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.065+0000 7fdb33eb4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2c13e380 con 0x7fdb2c15ed10 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.065+0000 7fdb33eb4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c15fc30 con 0x7fdb2c06e7e0 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.065+0000 7fdb32451640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdb2c0cfaf0 0x7fdb2c0cff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54860/0 (socket says 172.21.15.111:54860) 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.065+0000 7fdb32451640 1 -- 172.21.15.111:0/1641358098 learned_addr learned my addr 172.21.15.111:0/1641358098 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277964052 0 0) 0x7fdb2c06f100 con 0x7fdb2c0cfaf0 2023-12-04T05:39:56.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb31c50640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 0x7fdb2c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb18003520 con 0x7fdb2c0cfaf0 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3635692554 0 0) 0x7fdb18003520 con 0x7fdb2c0cfaf0 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 msgr2=0x7fdb2c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 0x7fdb2c06ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 >> v1:172.21.15.192:6789/0 conn(0x7fdb2c15ed10 legacy=0x7fdb2c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c15b810 con 0x7fdb2c0cfaf0 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fdb24001670 con 0x7fdb2c0cfaf0 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 >> v1:172.21.15.111:6789/0 conn(0x7fdb2c0cfaf0 legacy=0x7fdb2c0cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 0x7fdb180039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb18003f20 0x7fdb180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18004d00 0x7fdb18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c15fc30 con 0x7fdb18003f20 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800bb10 con 0x7fdb18004d00 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800bcd0 con 0x7fdb2c06e7e0 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb3144f640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb18003f20 0x7fdb180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb31c50640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 0x7fdb180039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb32451640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18004d00 0x7fdb18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb3144f640 1 -- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 msgr2=0x7fdb180039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb3144f640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c06e7e0 0x7fdb180039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb3144f640 1 -- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18004d00 msgr2=0x7fdb18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb3144f640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18004d00 0x7fdb18005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.066+0000 7fdb3144f640 1 -- 172.21.15.111:0/1641358098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18003520 con 0x7fdb18003f20 2023-12-04T05:39:56.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.067+0000 7fdb3144f640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb18003f20 0x7fdb180047c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c001720 tx=0x7fdb180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85238f89aebd2e5b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.067+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0052d0 con 0x7fdb18003f20 2023-12-04T05:39:56.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.067+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c02e050 con 0x7fdb18003f20 2023-12-04T05:39:56.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.067+0000 7fdb30c4e640 1 -- 172.21.15.111:0/1641358098 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0035f0 con 0x7fdb18003f20 2023-12-04T05:39:56.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb33eb4640 1 -- 172.21.15.111:0/1641358098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb18003f20 msgr2=0x7fdb180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/1641358098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb18003f20 0x7fdb180047c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c001720 tx=0x7fdb180057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb32451640 1 -- 172.21.15.111:0/1641358098 reap_dead start 2023-12-04T05:39:56.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb33eb4640 1 -- 172.21.15.111:0/1641358098 shutdown_connections 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb33eb4640 1 -- 172.21.15.111:0/1641358098 >> 172.21.15.111:0/1641358098 conn(0x7fdb2c06a8b0 msgr2=0x7fdb2c06acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb33eb4640 1 -- 172.21.15.111:0/1641358098 shutdown_connections 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.068+0000 7fdb33eb4640 1 -- 172.21.15.111:0/1641358098 wait complete. 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.069+0000 7fdb33eb4640 1 Processor -- start 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.069+0000 7fdb33eb4640 1 -- start start 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18003f20 0x7fdb2c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 0x7fdb2c1f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c1e6f20 0x7fdb2c1e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c1e78e0 con 0x7fdb2c1583f0 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c13f5f0 con 0x7fdb18003f20 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c1e6990 con 0x7fdb2c1e6f20 2023-12-04T05:39:56.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 0x7fdb2c1f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 0x7fdb2c1f0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36392/0 (socket says 172.21.15.111:36392) 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 -- 172.21.15.111:0/306310823 learned_addr learned my addr 172.21.15.111:0/306310823 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb31c50640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18003f20 0x7fdb2c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 -- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c1e6f20 msgr2=0x7fdb2c1e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c1e6f20 0x7fdb2c1e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 -- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18003f20 msgr2=0x7fdb2c157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18003f20 0x7fdb2c157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 -- 172.21.15.111:0/306310823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c13f840 con 0x7fdb2c1583f0 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb31c50640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18003f20 0x7fdb2c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb3144f640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 0x7fdb2c1f0fd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c008fd0 tx=0x7fdb1c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c007a30 con 0x7fdb2c1583f0 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c007640 con 0x7fdb2c1583f0 2023-12-04T05:39:56.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c13fb20 con 0x7fdb2c1583f0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c006c00 con 0x7fdb2c1583f0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.070+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c158190 con 0x7fdb2c1583f0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.071+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c0693d0 con 0x7fdb2c1583f0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.072+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fdb1c0035f0 con 0x7fdb2c1583f0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.072+0000 7fdb227fc640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb1006b2a0 0x7fdb1006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.072+0000 7fdb31c50640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb1006b2a0 0x7fdb1006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.072+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb1c0ab890 con 0x7fdb2c1583f0 2023-12-04T05:39:56.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.073+0000 7fdb31c50640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb1006b2a0 0x7fdb1006d760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007440 tx=0x7fdb2800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.076+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb1c076790 con 0x7fdb2c1583f0 2023-12-04T05:39:56.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.246+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb2c0b4010 con 0x7fdb2c1583f0 2023-12-04T05:39:56.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.247+0000 7fdb227fc640 1 -- 172.21.15.111:0/306310823 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fdb1c04b020 con 0x7fdb2c1583f0 2023-12-04T05:39:56.249 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:56.249 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:18.190369+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:56.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb1006b2a0 msgr2=0x7fdb1006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb1006b2a0 0x7fdb1006d760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007440 tx=0x7fdb2800b040 comp rx=0 tx=0).stop 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 msgr2=0x7fdb2c1f0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 0x7fdb2c1f0fd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c008fd0 tx=0x7fdb1c001120 comp rx=0 tx=0).stop 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 shutdown_connections 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb1006b2a0 0x7fdb1006d760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb2c1e6f20 0x7fdb2c1e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb2c1583f0 0x7fdb2c1f0fd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 --2- 172.21.15.111:0/306310823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb18003f20 0x7fdb2c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 >> 172.21.15.111:0/306310823 conn(0x7fdb2c06a8b0 msgr2=0x7fdb2c0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 shutdown_connections 2023-12-04T05:39:56.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.249+0000 7fdb33eb4640 1 -- 172.21.15.111:0/306310823 wait complete. 2023-12-04T05:39:56.261 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:56.262 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:56.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.364+0000 7fde9bba9640 1 Processor -- start 2023-12-04T05:39:56.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.364+0000 7fde9bba9640 1 -- start start 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde9bba9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 0x7fde94161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde9bba9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fde9415dfb0 0x7fde940d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde9bba9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde940d18b0 con 0x7fde940d0f10 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde9bba9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9406eaf0 con 0x7fde9415dfb0 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde9bba9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9415e4e0 con 0x7fde94146b10 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde99144640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fde940d0f10 0x7fde9415d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54866/0 (socket says 172.21.15.111:54866) 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde99144640 1 -- 172.21.15.111:0/3388487342 learned_addr learned my addr 172.21.15.111:0/3388487342 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889823758 0 0) 0x7fde940d18b0 con 0x7fde940d0f10 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde80003520 con 0x7fde940d0f10 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde99945640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 0x7fde94161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.365+0000 7fde9a146640 1 --2- 172.21.15.111:0/3388487342 >> v2:172.21.15.192:3300/0 conn(0x7fde9415dfb0 0x7fde940d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 224111656 0 0) 0x7fde80003520 con 0x7fde940d0f10 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 msgr2=0x7fde94161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 0x7fde94161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 >> v2:172.21.15.192:3300/0 conn(0x7fde9415dfb0 msgr2=0x7fde940d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 --2- 172.21.15.111:0/3388487342 >> v2:172.21.15.192:3300/0 conn(0x7fde9415dfb0 0x7fde940d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9415e7c0 con 0x7fde940d0f10 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fde84001670 con 0x7fde940d0f10 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 >> v1:172.21.15.111:6789/0 conn(0x7fde940d0f10 legacy=0x7fde9415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 0x7fde80003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde80003ef0 0x7fde800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde80004d00 0x7fde80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9406eaf0 con 0x7fde80004d00 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9415e4e0 con 0x7fde80003ef0 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8000b980 con 0x7fde94146b10 2023-12-04T05:39:56.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde9a146640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde80004d00 0x7fde80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde99945640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 0x7fde80003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde99144640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde80003ef0 0x7fde800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde9a146640 1 -- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 msgr2=0x7fde80003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde9a146640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde94146b10 0x7fde80003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde9a146640 1 -- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde80003ef0 msgr2=0x7fde800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde9a146640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde80003ef0 0x7fde800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.366+0000 7fde9a146640 1 -- 172.21.15.111:0/3388487342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80003520 con 0x7fde80004d00 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.367+0000 7fde9a146640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde80004d00 0x7fde80005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009ac0 tx=0x7fde800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0ebeb911dc7dfe server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.367+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c02c6d0 con 0x7fde80004d00 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.367+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c02c830 con 0x7fde80004d00 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.367+0000 7fde98943640 1 -- 172.21.15.111:0/3388487342 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c02cbc0 con 0x7fde80004d00 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9bba9640 1 -- 172.21.15.111:0/3388487342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde80004d00 msgr2=0x7fde80005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3388487342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde80004d00 0x7fde80005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009ac0 tx=0x7fde800057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9a146640 1 -- 172.21.15.111:0/3388487342 reap_dead start 2023-12-04T05:39:56.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9bba9640 1 -- 172.21.15.111:0/3388487342 shutdown_connections 2023-12-04T05:39:56.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9bba9640 1 -- 172.21.15.111:0/3388487342 >> 172.21.15.111:0/3388487342 conn(0x7fde94070af0 msgr2=0x7fde940662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9bba9640 1 -- 172.21.15.111:0/3388487342 shutdown_connections 2023-12-04T05:39:56.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.368+0000 7fde9bba9640 1 -- 172.21.15.111:0/3388487342 wait complete. 2023-12-04T05:39:56.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.369+0000 7fde9bba9640 1 Processor -- start 2023-12-04T05:39:56.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.369+0000 7fde9bba9640 1 -- start start 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde80004d00 0x7fde94159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 0x7fde941406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9413df20 0x7fde94140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde94159ac0 con 0x7fde9413daa0 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde940b2f00 con 0x7fde9413df20 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9413e4d0 con 0x7fde80004d00 2023-12-04T05:39:56.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 0x7fde941406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 0x7fde941406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36406/0 (socket says 172.21.15.111:36406) 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 -- 172.21.15.111:0/3304732049 learned_addr learned my addr 172.21.15.111:0/3304732049 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99945640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde80004d00 0x7fde94159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9a146640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9413df20 0x7fde94140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 -- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde80004d00 msgr2=0x7fde94159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde80004d00 0x7fde94159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 -- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9413df20 msgr2=0x7fde94140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9413df20 0x7fde94140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 -- 172.21.15.111:0/3304732049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde940b2140 con 0x7fde9413daa0 2023-12-04T05:39:56.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99945640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde80004d00 0x7fde94159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9a146640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9413df20 0x7fde94140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde99144640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 0x7fde941406e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fde840072c0 tx=0x7fde84007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400f920 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8400fa80 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde940b2420 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400fe30 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.370+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde940b29e0 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.371+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde94069360 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.372+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fde8401d460 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.372+0000 7fde8a7fc640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde7806b2d0 0x7fde7806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.372+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde84089e10 con 0x7fde9413daa0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.372+0000 7fde99945640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde7806b2d0 0x7fde7806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.372+0000 7fde99945640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde7806b2d0 0x7fde7806d790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde900074d0 tx=0x7fde9000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.376+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde84054c80 con 0x7fde9413daa0 2023-12-04T05:39:56.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.545+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde940b3950 con 0x7fde9413daa0 2023-12-04T05:39:56.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.545+0000 7fde8a7fc640 1 -- 172.21.15.111:0/3304732049 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde840582a0 con 0x7fde9413daa0 2023-12-04T05:39:56.547 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:56.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde7806b2d0 msgr2=0x7fde7806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde7806b2d0 0x7fde7806d790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde900074d0 tx=0x7fde9000b040 comp rx=0 tx=0).stop 2023-12-04T05:39:56.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 msgr2=0x7fde941406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 0x7fde941406e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fde840072c0 tx=0x7fde84007780 comp rx=0 tx=0).stop 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 shutdown_connections 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde7806b2d0 0x7fde7806d790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9413df20 0x7fde94140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde9413daa0 0x7fde941406e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 --2- 172.21.15.111:0/3304732049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde80004d00 0x7fde94159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 >> 172.21.15.111:0/3304732049 conn(0x7fde94070af0 msgr2=0x7fde94144980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 shutdown_connections 2023-12-04T05:39:56.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.547+0000 7fde9bba9640 1 -- 172.21.15.111:0/3304732049 wait complete. 2023-12-04T05:39:56.559 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:56.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.660+0000 7fad67e1c640 1 Processor -- start 2023-12-04T05:39:56.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.660+0000 7fad67e1c640 1 -- start start 2023-12-04T05:39:56.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad67e1c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad600af750 0x7fad600ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad67e1c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6015df30 con 0x7fad6015da80 2023-12-04T05:39:56.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad67e1c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6015e190 con 0x7fad600ac100 2023-12-04T05:39:56.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad67e1c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad60157f60 con 0x7fad600af750 2023-12-04T05:39:56.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad65bb8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad600af750 0x7fad600ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad653b7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fad6015da80 0x7fad600afb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54872/0 (socket says 172.21.15.111:54872) 2023-12-04T05:39:56.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad653b7640 1 -- 172.21.15.111:0/3783843365 learned_addr learned my addr 172.21.15.111:0/3783843365 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:56.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad65bb8640 1 -- 172.21.15.111:0/3783843365 >> v1:172.21.15.192:6789/0 conn(0x7fad600ac100 legacy=0x7fad600cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad65bb8640 1 -- 172.21.15.111:0/3783843365 >> v1:172.21.15.111:6789/0 conn(0x7fad6015da80 legacy=0x7fad600afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad65bb8640 1 -- 172.21.15.111:0/3783843365 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6015e470 con 0x7fad600af750 2023-12-04T05:39:56.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.661+0000 7fad64bb6640 1 -- 172.21.15.111:0/3783843365 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303726566 0 0) 0x7fad6015df30 con 0x7fad6015da80 2023-12-04T05:39:56.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad65bb8640 1 --2- 172.21.15.111:0/3783843365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad600ae600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fad58009d90 tx=0x7fad5802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05b39c2e8f0e1a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad64bb6640 1 -- 172.21.15.111:0/3783843365 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5802cc60 con 0x7fad600af750 2023-12-04T05:39:56.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad64bb6640 1 -- 172.21.15.111:0/3783843365 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5802cdc0 con 0x7fad600af750 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad64bb6640 1 -- 172.21.15.111:0/3783843365 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58034660 con 0x7fad600af750 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad67e1c640 1 -- 172.21.15.111:0/3783843365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 msgr2=0x7fad600ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3783843365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad600ae600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fad58009d90 tx=0x7fad5802beb0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad67e1c640 1 -- 172.21.15.111:0/3783843365 shutdown_connections 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3783843365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad600ae600 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.662+0000 7fad67e1c640 1 -- 172.21.15.111:0/3783843365 >> 172.21.15.111:0/3783843365 conn(0x7fad600b4640 msgr2=0x7fad600b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.663+0000 7fad67e1c640 1 -- 172.21.15.111:0/3783843365 shutdown_connections 2023-12-04T05:39:56.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.663+0000 7fad67e1c640 1 -- 172.21.15.111:0/3783843365 wait complete. 2023-12-04T05:39:56.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 Processor -- start 2023-12-04T05:39:56.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 -- start start 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600ac100 0x7fad6014a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad6009ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad6015da80 0x7fad6009bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6009c7d0 con 0x7fad600af750 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6009cab0 con 0x7fad6015da80 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.664+0000 7fad67e1c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6009cd90 con 0x7fad600ac100 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad6009ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad6009ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36424/0 (socket says 172.21.15.111:36424) 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 -- 172.21.15.111:0/3606801323 learned_addr learned my addr 172.21.15.111:0/3606801323 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad65bb8640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600ac100 0x7fad6014a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad663b9640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad6015da80 0x7fad6009bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 -- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600ac100 msgr2=0x7fad6014a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600ac100 0x7fad6014a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 -- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad6015da80 msgr2=0x7fad6009bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad6015da80 0x7fad6009bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 -- 172.21.15.111:0/3606801323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58009a70 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad653b7640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad6009ba00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c3c0 tx=0x7fad5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50005ce0 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad50005e40 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6009d070 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50007310 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.665+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6009c480 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.666+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad600c9750 con 0x7fad600af750 2023-12-04T05:39:56.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.666+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fad50007530 con 0x7fad600af750 2023-12-04T05:39:56.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.667+0000 7fad567fc640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad4406b2a0 0x7fad4406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.667+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad500890f0 con 0x7fad600af750 2023-12-04T05:39:56.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.667+0000 7fad65bb8640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad4406b2a0 0x7fad4406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.667+0000 7fad65bb8640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad4406b2a0 0x7fad4406d760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad58036000 tx=0x7fad58030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.671+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad600c9750 con 0x7fad600af750 2023-12-04T05:39:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.836+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad600c9750 con 0x7fad600af750 2023-12-04T05:39:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.837+0000 7fad567fc640 1 -- 172.21.15.111:0/3606801323 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad600c9750 con 0x7fad600af750 2023-12-04T05:39:56.838 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad4406b2a0 msgr2=0x7fad4406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad4406b2a0 0x7fad4406d760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad58036000 tx=0x7fad58030000 comp rx=0 tx=0).stop 2023-12-04T05:39:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 msgr2=0x7fad6009ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad6009ba00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c3c0 tx=0x7fad5000c880 comp rx=0 tx=0).stop 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 shutdown_connections 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad4406b2a0 0x7fad4406d760 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad6015da80 0x7fad6009bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad600af750 0x7fad6009ba00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 --2- 172.21.15.111:0/3606801323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600ac100 0x7fad6014a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 >> 172.21.15.111:0/3606801323 conn(0x7fad600b4640 msgr2=0x7fad6006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 shutdown_connections 2023-12-04T05:39:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.839+0000 7fad67e1c640 1 -- 172.21.15.111:0/3606801323 wait complete. 2023-12-04T05:39:56.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:56.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.952+0000 7f8a61d53640 1 Processor -- start 2023-12-04T05:39:56.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.952+0000 7f8a61d53640 1 -- start start 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a61d53640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a5c13cee0 0x7f8a5c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a61d53640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c13f640 0x7f8a5c13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a61d53640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c13d2c0 con 0x7f8a5c071d20 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a61d53640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c1400e0 con 0x7f8a5c13f640 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a61d53640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c1403c0 con 0x7f8a5c13cee0 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c13f640 0x7f8a5c13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c13f640 0x7f8a5c13fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36426/0 (socket says 172.21.15.111:36426) 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 -- 172.21.15.111:0/3167013236 learned_addr learned my addr 172.21.15.111:0/3167013236 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/3167013236 >> v2:172.21.15.192:3300/0 conn(0x7f8a5c13cee0 0x7f8a5c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 -- 172.21.15.111:0/3167013236 >> v2:172.21.15.192:3300/0 conn(0x7f8a5c13cee0 msgr2=0x7f8a5c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 --2- 172.21.15.111:0/3167013236 >> v2:172.21.15.192:3300/0 conn(0x7f8a5c13cee0 0x7f8a5c13d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 -- 172.21.15.111:0/3167013236 >> v1:172.21.15.192:6789/0 conn(0x7f8a5c071d20 legacy=0x7f8a5c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.953+0000 7f8a5affd640 1 -- 172.21.15.111:0/3167013236 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0cdb50 con 0x7f8a5c13f640 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.954+0000 7f8a5affd640 1 --2- 172.21.15.111:0/3167013236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c13fab0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c06b470 tx=0x7f8a5002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e17171fe39dcfc server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.954+0000 7f8a5a7fc640 1 -- 172.21.15.111:0/3167013236 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50031750 con 0x7f8a5c13f640 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.954+0000 7f8a5a7fc640 1 -- 172.21.15.111:0/3167013236 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500318b0 con 0x7f8a5c13f640 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.954+0000 7f8a5a7fc640 1 -- 172.21.15.111:0/3167013236 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50031c40 con 0x7f8a5c13f640 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 -- 172.21.15.111:0/3167013236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 msgr2=0x7f8a5c13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 --2- 172.21.15.111:0/3167013236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c13fab0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c06b470 tx=0x7f8a5002b7f0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 -- 172.21.15.111:0/3167013236 shutdown_connections 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 --2- 172.21.15.111:0/3167013236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c13fab0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 --2- 172.21.15.111:0/3167013236 >> v2:172.21.15.192:3300/0 conn(0x7f8a5c13cee0 0x7f8a5c13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 -- 172.21.15.111:0/3167013236 >> 172.21.15.111:0/3167013236 conn(0x7f8a5c0b0cf0 msgr2=0x7f8a5c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 -- 172.21.15.111:0/3167013236 shutdown_connections 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.955+0000 7f8a61d53640 1 -- 172.21.15.111:0/3167013236 wait complete. 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.956+0000 7f8a61d53640 1 Processor -- start 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.956+0000 7f8a61d53640 1 -- start start 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a5c071d20 0x7f8a5c138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a5c13cee0 0x7f8a5c161a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c15d5d0 con 0x7f8a5c13f640 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c162500 con 0x7f8a5c13cee0 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c15e950 con 0x7f8a5c071d20 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c161fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36436/0 (socket says 172.21.15.111:36436) 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 -- 172.21.15.111:0/2303185150 learned_addr learned my addr 172.21.15.111:0/2303185150 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a5c13cee0 0x7f8a5c161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a5c071d20 0x7f8a5c138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 -- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a5c071d20 msgr2=0x7f8a5c138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a5c071d20 0x7f8a5c138630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 -- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a5c13cee0 msgr2=0x7f8a5c161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a5c13cee0 0x7f8a5c161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 -- 172.21.15.111:0/2303185150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a500094d0 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a5bfff640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c161fc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c7b0 tx=0x7f8a5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54004b60 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a54004cc0 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5c15ec30 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.957+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400aa90 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.958+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c15f1f0 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.958+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5c069360 con 0x7f8a5c13f640 2023-12-04T05:39:56.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.959+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8a5400abf0 con 0x7f8a5c13f640 2023-12-04T05:39:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.959+0000 7f8a43fff640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8a4806b2a0 0x7f8a4806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.959+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a54089d10 con 0x7f8a5c13f640 2023-12-04T05:39:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.959+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8a4806b2a0 0x7f8a4806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.960+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8a4806b2a0 0x7f8a4806d760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006fd0 tx=0x7f8a4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:56.963+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a54054b80 con 0x7f8a5c13f640 2023-12-04T05:39:57.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.131+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a5c070380 con 0x7f8a5c13f640 2023-12-04T05:39:57.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.132+0000 7f8a43fff640 1 -- 172.21.15.111:0/2303185150 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a540581a0 con 0x7f8a5c13f640 2023-12-04T05:39:57.133 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8a4806b2a0 msgr2=0x7f8a4806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8a4806b2a0 0x7f8a4806d760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006fd0 tx=0x7f8a4c008040 comp rx=0 tx=0).stop 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 msgr2=0x7f8a5c161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c161fc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c7b0 tx=0x7f8a5400cc70 comp rx=0 tx=0).stop 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 shutdown_connections 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8a4806b2a0 0x7f8a4806d760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c13f640 0x7f8a5c161fc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a5c13cee0 0x7f8a5c161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 --2- 172.21.15.111:0/2303185150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a5c071d20 0x7f8a5c138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 >> 172.21.15.111:0/2303185150 conn(0x7f8a5c0b0cf0 msgr2=0x7f8a5c159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:57.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 shutdown_connections 2023-12-04T05:39:57.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.134+0000 7f8a61d53640 1 -- 172.21.15.111:0/2303185150 wait complete. 2023-12-04T05:39:57.146 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:39:57.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.248+0000 7f760ec15640 1 Processor -- start 2023-12-04T05:39:57.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.248+0000 7f760ec15640 1 -- start start 2023-12-04T05:39:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760ec15640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7608144430 0x7f760815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760ec15640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7608070bd0 con 0x7f760806c250 2023-12-04T05:39:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760ec15640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7608070eb0 con 0x7f76080703f0 2023-12-04T05:39:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760ec15640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7608144810 con 0x7f7608144430 2023-12-04T05:39:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760d1b2640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f760806c250 0x7f760815b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54878/0 (socket says 172.21.15.111:54878) 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760d1b2640 1 -- 172.21.15.111:0/4141429714 learned_addr learned my addr 172.21.15.111:0/4141429714 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f760c9b1640 1 --2- 172.21.15.111:0/4141429714 >> v2:172.21.15.192:3300/0 conn(0x7f7608144430 0x7f760815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866212510 0 0) 0x7f7608070bd0 con 0x7f760806c250 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f0003520 con 0x7f760806c250 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194332447 0 0) 0x7f75f0003520 con 0x7f760806c250 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 >> v2:172.21.15.192:3300/0 conn(0x7f7608144430 msgr2=0x7f760815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 --2- 172.21.15.111:0/4141429714 >> v2:172.21.15.192:3300/0 conn(0x7f7608144430 0x7f760815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 >> v1:172.21.15.192:6789/0 conn(0x7f76080703f0 legacy=0x7f7608071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7608071190 con 0x7f760806c250 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.249+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f76000019f0 con 0x7f760806c250 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 >> v1:172.21.15.111:6789/0 conn(0x7f760806c250 legacy=0x7f760815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75f0003970 0x7f75f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75f00047c0 0x7f75f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f75f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7608144810 con 0x7f75f0005180 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f000c020 con 0x7f75f0003970 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f000c1e0 con 0x7f75f00047c0 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760d1b2640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f75f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760c9b1640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75f0003970 0x7f75f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f7607fff640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75f00047c0 0x7f75f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760d1b2640 1 -- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75f00047c0 msgr2=0x7f75f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760d1b2640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75f00047c0 0x7f75f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760d1b2640 1 -- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75f0003970 msgr2=0x7f75f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760d1b2640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75f0003970 0x7f75f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.250+0000 7f760d1b2640 1 -- 172.21.15.111:0/4141429714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f0003520 con 0x7f75f0005180 2023-12-04T05:39:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760d1b2640 1 -- 172.21.15.111:0/4141429714 reap_dead start 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760d1b2640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f75f000b980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f760000a830 tx=0x7f75f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba1d1a2cb0f6d4d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76000072d0 con 0x7f75f0005180 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760002f070 con 0x7f75f0005180 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f76077fe640 1 -- 172.21.15.111:0/4141429714 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76000076b0 con 0x7f75f0005180 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 -- 172.21.15.111:0/4141429714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 msgr2=0x7f75f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f75f000b980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f760000a830 tx=0x7f75f00057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 -- 172.21.15.111:0/4141429714 shutdown_connections 2023-12-04T05:39:57.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 --2- 172.21.15.111:0/4141429714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f75f000b980 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 -- 172.21.15.111:0/4141429714 >> 172.21.15.111:0/4141429714 conn(0x7f760806a4c0 msgr2=0x7f760806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 -- 172.21.15.111:0/4141429714 shutdown_connections 2023-12-04T05:39:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.251+0000 7f760ec15640 1 -- 172.21.15.111:0/4141429714 wait complete. 2023-12-04T05:39:57.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 Processor -- start 2023-12-04T05:39:57.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 -- start start 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f760815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76080c9c90 0x7f76080ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76080cae60 0x7f76080a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760815afd0 con 0x7f75f0005180 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76080c8bf0 con 0x7f76080c9c90 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.253+0000 7f760ec15640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76080c8ed0 con 0x7f76080cae60 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f760815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f760815aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36448/0 (socket says 172.21.15.111:36448) 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 -- 172.21.15.111:0/863958830 learned_addr learned my addr 172.21.15.111:0/863958830 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f7607fff640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76080c9c90 0x7f76080ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760d1b2640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76080cae60 0x7f76080a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 -- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76080cae60 msgr2=0x7f76080a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76080cae60 0x7f76080a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 -- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76080c9c90 msgr2=0x7f76080ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76080c9c90 0x7f76080ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 -- 172.21.15.111:0/863958830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76080c91b0 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760d1b2640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76080cae60 0x7f76080a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f7607fff640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76080c9c90 0x7f76080ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760c9b1640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f760815aa90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f75f80118e0 tx=0x7f75f8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800faf0 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f800fc50 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76080c9490 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8014440 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.254+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76080ca4d0 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.255+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760806d100 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.256+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f75f801d400 con 0x7f75f0005180 2023-12-04T05:39:57.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.256+0000 7f76057fa640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f75f406b2a0 0x7f75f406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.256+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75f808a4b0 con 0x7f75f0005180 2023-12-04T05:39:57.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.256+0000 7f7607fff640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f75f406b2a0 0x7f75f406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.256+0000 7f7607fff640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f75f406b2a0 0x7f75f406d760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f760815c570 tx=0x7f75fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.260+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75f8055320 con 0x7f75f0005180 2023-12-04T05:39:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.446+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f76080693b0 con 0x7f75f0005180 2023-12-04T05:39:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.446+0000 7f76057fa640 1 -- 172.21.15.111:0/863958830 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f75f8058940 con 0x7f75f0005180 2023-12-04T05:39:57.447 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:39:57.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f75f406b2a0 msgr2=0x7f75f406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f75f406b2a0 0x7f75f406d760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f760815c570 tx=0x7f75fc009040 comp rx=0 tx=0).stop 2023-12-04T05:39:57.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 msgr2=0x7f760815aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f760815aa90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f75f80118e0 tx=0x7f75f8011da0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 shutdown_connections 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f75f406b2a0 0x7f75f406d760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76080cae60 0x7f76080a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76080c9c90 0x7f76080ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 --2- 172.21.15.111:0/863958830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f0005180 0x7f760815aa90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 >> 172.21.15.111:0/863958830 conn(0x7f760806a4c0 msgr2=0x7f760815c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 shutdown_connections 2023-12-04T05:39:57.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.448+0000 7f760ec15640 1 -- 172.21.15.111:0/863958830 wait complete. 2023-12-04T05:39:57.460 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-04T05:39:57.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:57.460 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-12-04T05:39:57.492 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:39:57.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:39:57.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:39:57.492 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-04T05:39:57.549 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-04T05:39:57.549 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:57.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.648+0000 7f0ba301e640 1 Processor -- start 2023-12-04T05:39:57.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.648+0000 7f0ba301e640 1 -- start start 2023-12-04T05:39:57.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba301e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b9c15c060 0x7f0b9c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba301e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c06b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba301e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c0cd240 con 0x7f0b9c151890 2023-12-04T05:39:57.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba301e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c06be90 con 0x7f0b9c15c060 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba301e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0cd520 con 0x7f0b9c06b4d0 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0b9bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c06b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba0dba640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b9c15c060 0x7f0b9c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba15bb640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f0b9c151890 0x7f0b9c151d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50910/0 (socket says 172.21.15.111:50910) 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba15bb640 1 -- 172.21.15.111:0/1928596745 learned_addr learned my addr 172.21.15.111:0/1928596745 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba0dba640 1 -- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 msgr2=0x7f0b9c06b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba0dba640 1 --2- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c06b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.649+0000 7f0ba0dba640 1 -- 172.21.15.111:0/1928596745 >> v1:172.21.15.192:6789/0 conn(0x7f0b9c151890 legacy=0x7f0b9c151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.650+0000 7f0ba0dba640 1 -- 172.21.15.111:0/1928596745 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0cd800 con 0x7f0b9c15c060 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.650+0000 7f0b9bfff640 1 --2- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c06b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:57.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.650+0000 7f0ba0dba640 1 --2- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0cc8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0b900097a0 tx=0x7f0b9002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968228631c4cbb8b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.650+0000 7f0b9b7fe640 1 -- 172.21.15.111:0/1928596745 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9002e070 con 0x7f0b9c15c060 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0b9b7fe640 1 -- 172.21.15.111:0/1928596745 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b900315a0 con 0x7f0b9c15c060 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0b9b7fe640 1 -- 172.21.15.111:0/1928596745 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90031930 con 0x7f0b9c15c060 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 -- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 msgr2=0x7f0b9c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 --2- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0cc8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0b900097a0 tx=0x7f0b9002b840 comp rx=0 tx=0).stop 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 -- 172.21.15.111:0/1928596745 shutdown_connections 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 --2- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c06b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 --2- 172.21.15.111:0/1928596745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0cc8d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 -- 172.21.15.111:0/1928596745 >> 172.21.15.111:0/1928596745 conn(0x7f0b9c0b1b50 msgr2=0x7f0b9c070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 -- 172.21.15.111:0/1928596745 shutdown_connections 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.651+0000 7f0ba301e640 1 -- 172.21.15.111:0/1928596745 wait complete. 2023-12-04T05:39:57.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.652+0000 7f0ba301e640 1 Processor -- start 2023-12-04T05:39:57.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.652+0000 7f0ba301e640 1 -- start start 2023-12-04T05:39:57.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba301e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c1f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba301e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 0x7f0b9c0c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba301e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba301e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c4cd0 con 0x7f0b9c151890 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba301e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0abe00 con 0x7f0b9c15c060 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba301e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c145b90 con 0x7f0b9c06b4d0 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0b9bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 0x7f0b9c0c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0b9bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 0x7f0b9c0c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36462/0 (socket says 172.21.15.111:36462) 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0b9bfff640 1 -- 172.21.15.111:0/3128906143 learned_addr learned my addr 172.21.15.111:0/3128906143 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba15bb640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.653+0000 7f0ba0dba640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c1f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b9bfff640 1 -- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 msgr2=0x7f0b9c1f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b9bfff640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c1f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b9bfff640 1 -- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 msgr2=0x7f0b9c0c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b9bfff640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b9bfff640 1 -- 172.21.15.111:0/3128906143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88009400 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0ba15bb640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b9bfff640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 0x7f0b9c0c4790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800e990 tx=0x7f0b8800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800c9e0 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8800cb40 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b90009400 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800ced0 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.654+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c0ab4d0 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.655+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c0693d0 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.655+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0b88013470 con 0x7f0b9c151890 2023-12-04T05:39:57.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.656+0000 7f0b997fa640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b8006b2d0 0x7f0b8006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.656+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0b88089f00 con 0x7f0b9c151890 2023-12-04T05:39:57.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.656+0000 7f0ba0dba640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b8006b2d0 0x7f0b8006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.656+0000 7f0ba0dba640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b8006b2d0 0x7f0b8006d790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b90036000 tx=0x7f0b9003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.660+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b88054cf0 con 0x7f0b9c151890 2023-12-04T05:39:57.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.861+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b9c0717f0 con 0x7f0b9c151890 2023-12-04T05:39:57.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.861+0000 7f0b997fa640 1 -- 172.21.15.111:0/3128906143 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f0b88017090 con 0x7f0b9c151890 2023-12-04T05:39:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:57.863 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:57.863 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:57.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.863+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b8006b2d0 msgr2=0x7f0b8006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.863+0000 7f0ba301e640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b8006b2d0 0x7f0b8006d790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b90036000 tx=0x7f0b9003d040 comp rx=0 tx=0).stop 2023-12-04T05:39:57.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.863+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 msgr2=0x7f0b9c0c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.863+0000 7f0ba301e640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 0x7f0b9c0c4790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800e990 tx=0x7f0b8800ee50 comp rx=0 tx=0).stop 2023-12-04T05:39:57.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 shutdown_connections 2023-12-04T05:39:57.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b8006b2d0 0x7f0b8006d790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b9c15c060 0x7f0b9c0c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c151890 0x7f0b9c0c4790 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 --2- 172.21.15.111:0/3128906143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b9c06b4d0 0x7f0b9c1f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 >> 172.21.15.111:0/3128906143 conn(0x7f0b9c0b1b50 msgr2=0x7f0b9c06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:57.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 shutdown_connections 2023-12-04T05:39:57.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.864+0000 7f0ba301e640 1 -- 172.21.15.111:0/3128906143 wait complete. 2023-12-04T05:39:57.876 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:57.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.974+0000 7f4b19ba9640 1 Processor -- start 2023-12-04T05:39:57.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.974+0000 7f4b19ba9640 1 -- start start 2023-12-04T05:39:57.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b19ba9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b141544c0 0x7f4b1406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b19ba9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b1406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b19ba9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4b1406ba10 0x7f4b14160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b19ba9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1406bec0 con 0x7f4b141544c0 2023-12-04T05:39:57.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b19ba9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b140ad5d0 con 0x7f4b1406ba10 2023-12-04T05:39:57.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b19ba9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1406df90 con 0x7f4b1406d250 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b141544c0 0x7f4b1406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b141544c0 0x7f4b1406cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36468/0 (socket says 172.21.15.111:36468) 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 -- 172.21.15.111:0/2559625386 learned_addr learned my addr 172.21.15.111:0/2559625386 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b12ffd640 1 --2- 172.21.15.111:0/2559625386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b1406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b13fff640 1 --2- 172.21.15.111:0/2559625386 >> v2:172.21.15.192:3300/0 conn(0x7f4b1406ba10 0x7f4b14160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 -- 172.21.15.111:0/2559625386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 msgr2=0x7f4b1406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 --2- 172.21.15.111:0/2559625386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b1406b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 -- 172.21.15.111:0/2559625386 >> v2:172.21.15.192:3300/0 conn(0x7f4b1406ba10 msgr2=0x7f4b14160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 --2- 172.21.15.111:0/2559625386 >> v2:172.21.15.192:3300/0 conn(0x7f4b1406ba10 0x7f4b14160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.975+0000 7f4b137fe640 1 -- 172.21.15.111:0/2559625386 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b141548a0 con 0x7f4b141544c0 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b137fe640 1 --2- 172.21.15.111:0/2559625386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b1406cd10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b1406c0a0 tx=0x7f4b0402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af25abb0e9f8c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b127fc640 1 -- 172.21.15.111:0/2559625386 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04031870 con 0x7f4b141544c0 2023-12-04T05:39:57.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b127fc640 1 -- 172.21.15.111:0/2559625386 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b040319d0 con 0x7f4b141544c0 2023-12-04T05:39:57.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b127fc640 1 -- 172.21.15.111:0/2559625386 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04031d60 con 0x7f4b141544c0 2023-12-04T05:39:57.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 -- 172.21.15.111:0/2559625386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 msgr2=0x7f4b1406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/2559625386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b1406cd10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b1406c0a0 tx=0x7f4b0402b910 comp rx=0 tx=0).stop 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 -- 172.21.15.111:0/2559625386 shutdown_connections 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/2559625386 >> v2:172.21.15.192:3300/0 conn(0x7f4b1406ba10 0x7f4b14160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/2559625386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b1406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/2559625386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b1406cd10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 -- 172.21.15.111:0/2559625386 >> 172.21.15.111:0/2559625386 conn(0x7f4b140b1b50 msgr2=0x7f4b14070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 -- 172.21.15.111:0/2559625386 shutdown_connections 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.976+0000 7f4b19ba9640 1 -- 172.21.15.111:0/2559625386 wait complete. 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 Processor -- start 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 -- start start 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b1406ba10 0x7f4b1415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b14155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b14155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141561e0 con 0x7f4b141544c0 2023-12-04T05:39:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141564c0 con 0x7f4b1406ba10 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.978+0000 7f4b19ba9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141567a0 con 0x7f4b1406d250 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b14155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b14155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36482/0 (socket says 172.21.15.111:36482) 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 -- 172.21.15.111:0/4146926374 learned_addr learned my addr 172.21.15.111:0/4146926374 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b137fe640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b1406ba10 0x7f4b1415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b12ffd640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b14155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 -- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 msgr2=0x7f4b14155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b14155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 -- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b1406ba10 msgr2=0x7f4b1415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b1406ba10 0x7f4b1415cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 -- 172.21.15.111:0/4146926374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b040094d0 con 0x7f4b141544c0 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b13fff640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b14155bb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c007e60 tx=0x7f4b0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c014620 con 0x7f4b141544c0 2023-12-04T05:39:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00ee90 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b14156a80 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0149e0 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.979+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b141525a0 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.980+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b140693d0 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.980+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4b0c014b40 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.981+0000 7f4afbfff640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4b0006b2a0 0x7f4b0006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.981+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4b0c08a400 con 0x7f4b141544c0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.981+0000 7f4b137fe640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4b0006b2a0 0x7f4b0006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.981+0000 7f4b137fe640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4b0006b2a0 0x7f4b0006d760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b04036000 tx=0x7f4b0403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:57.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:57.985+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b0c0552a0 con 0x7f4b141544c0 2023-12-04T05:39:58.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.152+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b14071bf0 con 0x7f4b141544c0 2023-12-04T05:39:58.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.153+0000 7f4afbfff640 1 -- 172.21.15.111:0/4146926374 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16674 (secure 0 0 0) 0x7f4b0c0588c0 con 0x7f4b141544c0 2023-12-04T05:39:58.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:58.155 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":21,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:57.331869+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:58.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4b0006b2a0 msgr2=0x7f4b0006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4b0006b2a0 0x7f4b0006d760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b04036000 tx=0x7f4b0403c040 comp rx=0 tx=0).stop 2023-12-04T05:39:58.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 msgr2=0x7f4b14155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b14155bb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c007e60 tx=0x7f4b0c00e4b0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 shutdown_connections 2023-12-04T05:39:58.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4b0006b2a0 0x7f4b0006d760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b141544c0 0x7f4b14155bb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b1406d250 0x7f4b14155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 --2- 172.21.15.111:0/4146926374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b1406ba10 0x7f4b1415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.155+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 >> 172.21.15.111:0/4146926374 conn(0x7f4b140b1b50 msgr2=0x7f4b1406b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:58.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.156+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 shutdown_connections 2023-12-04T05:39:58.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.156+0000 7f4b19ba9640 1 -- 172.21.15.111:0/4146926374 wait complete. 2023-12-04T05:39:58.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:39:14.548588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:39:15.521726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:39:58.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:39:58.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.270+0000 7f9ba7fff640 1 Processor -- start 2023-12-04T05:39:58.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.270+0000 7f9ba7fff640 1 -- start start 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9ba815d480 0x7f9ba80d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9ba80ce6a0 0x7f9ba80d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba7fff640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba815d860 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba7fff640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba80b00d0 con 0x7f9ba815d480 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba7fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba80cebd0 con 0x7f9ba80ce6a0 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba6ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9ba815d480 0x7f9ba80d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9ba80b1eb0 0x7f9ba80cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54888/0 (socket says 172.21.15.111:54888) 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba77fe640 1 -- 172.21.15.111:0/1345826 learned_addr learned my addr 172.21.15.111:0/1345826 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9b9ffff640 1 --2- 172.21.15.111:0/1345826 >> v2:172.21.15.192:3300/0 conn(0x7f9ba80ce6a0 0x7f9ba80d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/1345826 >> v2:172.21.15.192:3300/0 conn(0x7f9ba80ce6a0 msgr2=0x7f9ba80d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba6ffd640 1 --2- 172.21.15.111:0/1345826 >> v2:172.21.15.192:3300/0 conn(0x7f9ba80ce6a0 0x7f9ba80d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/1345826 >> v1:172.21.15.111:6789/0 conn(0x7f9ba80b1eb0 legacy=0x7f9ba80cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/1345826 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80ceeb0 con 0x7f9ba815d480 2023-12-04T05:39:58.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.271+0000 7f9ba67fc640 1 -- 172.21.15.111:0/1345826 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446252241 0 0) 0x7f9ba0001490 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba6ffd640 1 --2- 172.21.15.111:0/1345826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba815d480 0x7f9ba80d1590 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80d0b30 tx=0x7f9b9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc0bc53f017d89d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba67fc640 1 -- 172.21.15.111:0/1345826 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9802e070 con 0x7f9ba815d480 2023-12-04T05:39:58.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba67fc640 1 -- 172.21.15.111:0/1345826 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b980315a0 con 0x7f9ba815d480 2023-12-04T05:39:58.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba67fc640 1 -- 172.21.15.111:0/1345826 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98031930 con 0x7f9ba815d480 2023-12-04T05:39:58.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba7fff640 1 -- 172.21.15.111:0/1345826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba815d480 msgr2=0x7f9ba80d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/1345826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba815d480 0x7f9ba80d1590 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80d0b30 tx=0x7f9b9802b840 comp rx=0 tx=0).stop 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba7fff640 1 -- 172.21.15.111:0/1345826 shutdown_connections 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/1345826 >> v2:172.21.15.192:3300/0 conn(0x7f9ba80ce6a0 0x7f9ba80d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/1345826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba815d480 0x7f9ba80d1590 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.272+0000 7f9ba7fff640 1 -- 172.21.15.111:0/1345826 >> 172.21.15.111:0/1345826 conn(0x7f9ba809c2f0 msgr2=0x7f9ba80d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.273+0000 7f9ba7fff640 1 -- 172.21.15.111:0/1345826 shutdown_connections 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.273+0000 7f9ba7fff640 1 -- 172.21.15.111:0/1345826 wait complete. 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.274+0000 7f9ba7fff640 1 Processor -- start 2023-12-04T05:39:58.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.274+0000 7f9ba7fff640 1 -- start start 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 0x7f9ba80ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ba80ce6a0 0x7f9ba80ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ba815d480 0x7f9ba809cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80cd2a0 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80b46a0 con 0x7f9ba80ce6a0 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80b3ad0 con 0x7f9ba815d480 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 0x7f9ba80ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 0x7f9ba80ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36500/0 (socket says 172.21.15.111:36500) 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/3872888078 learned_addr learned my addr 172.21.15.111:0/3872888078 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9b9ffff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ba80ce6a0 0x7f9ba80ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba77fe640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ba815d480 0x7f9ba809cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ba815d480 msgr2=0x7f9ba809cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ba815d480 0x7f9ba809cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ba80ce6a0 msgr2=0x7f9ba80ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ba80ce6a0 0x7f9ba80ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 -- 172.21.15.111:0/3872888078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98009400 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba77fe640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ba815d480 0x7f9ba809cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba6ffd640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 0x7f9ba80ccd60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009890 tx=0x7f9b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98032840 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b980329a0 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba80b3d70 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98032d50 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.275+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba80b4330 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.276+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba80a0f60 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.277+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9b9803b3f0 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.277+0000 7f9b9f7fe640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9b8806b2d0 0x7f9b8806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.277+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9b980abb90 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.277+0000 7f9b9ffff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9b8806b2d0 0x7f9b8806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.278+0000 7f9b9ffff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9b8806b2d0 0x7f9b8806d790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80a0eb0 tx=0x7f9b90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.281+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b98076a10 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.451+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ba808c770 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.451+0000 7f9b9f7fe640 1 -- 172.21.15.111:0/3872888078 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b9807a030 con 0x7f9ba80b1eb0 2023-12-04T05:39:58.453 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:39:58.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9b8806b2d0 msgr2=0x7f9b8806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9b8806b2d0 0x7f9b8806d790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80a0eb0 tx=0x7f9b90008040 comp rx=0 tx=0).stop 2023-12-04T05:39:58.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 msgr2=0x7f9ba80ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 0x7f9ba80ccd60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009890 tx=0x7f9b980046c0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 shutdown_connections 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9b8806b2d0 0x7f9b8806d790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ba815d480 0x7f9ba809cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ba80ce6a0 0x7f9ba80ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 --2- 172.21.15.111:0/3872888078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ba80b1eb0 0x7f9ba80ccd60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 >> 172.21.15.111:0/3872888078 conn(0x7f9ba809c2f0 msgr2=0x7f9ba80b76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 shutdown_connections 2023-12-04T05:39:58.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.453+0000 7f9ba7fff640 1 -- 172.21.15.111:0/3872888078 wait complete. 2023-12-04T05:39:58.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:39:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.568+0000 7f6186290640 1 Processor -- start 2023-12-04T05:39:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.568+0000 7f6186290640 1 -- start start 2023-12-04T05:39:58.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.568+0000 7f6186290640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 0x7f61800ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.568+0000 7f6186290640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61800cd320 con 0x7f6180072250 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.568+0000 7f6186290640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61800cc710 con 0x7f618015b0c0 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.568+0000 7f6186290640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618015f1a0 con 0x7f618015ed10 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6180072250 0x7f618015b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54900/0 (socket says 172.21.15.111:54900) 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617f7fe640 1 -- 172.21.15.111:0/2504804299 learned_addr learned my addr 172.21.15.111:0/2504804299 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848925967 0 0) 0x7f61800cd320 con 0x7f6180072250 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6168003520 con 0x7f6180072250 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f618482d640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 0x7f61800ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26292873 0 0) 0x7f6168003520 con 0x7f6180072250 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 msgr2=0x7f61800ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 0x7f61800ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 >> v1:172.21.15.192:6789/0 conn(0x7f618015b0c0 legacy=0x7f6180071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800cc9f0 con 0x7f6180072250 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f6174001670 con 0x7f6180072250 2023-12-04T05:39:58.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 >> v1:172.21.15.111:6789/0 conn(0x7f6180072250 legacy=0x7f618015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61680039a0 0x7f6168003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61680047c0 0x7f6168004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 0x7f61680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618015f1a0 con 0x7f61680047c0 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616800bab0 con 0x7f61680039a0 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616800bc70 con 0x7f618015ed10 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617f7fe640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61680047c0 0x7f6168004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f617ffff640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61680039a0 0x7f6168003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.569+0000 7f618482d640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 0x7f61680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617f7fe640 1 -- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 msgr2=0x7f61680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617f7fe640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f618015ed10 0x7f61680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617f7fe640 1 -- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61680039a0 msgr2=0x7f6168003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617f7fe640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61680039a0 0x7f6168003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617f7fe640 1 -- 172.21.15.111:0/2504804299 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168003520 con 0x7f61680047c0 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617ffff640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61680039a0 0x7f6168003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617f7fe640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61680047c0 0x7f6168004c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f617400c240 tx=0x7f61680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db8039441076d28 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174005ad0 con 0x7f61680047c0 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174005c30 con 0x7f61680047c0 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.570+0000 7f617effd640 1 -- 172.21.15.111:0/2504804299 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617402c8b0 con 0x7f61680047c0 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f6186290640 1 -- 172.21.15.111:0/2504804299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61680047c0 msgr2=0x7f6168004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f6186290640 1 --2- 172.21.15.111:0/2504804299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61680047c0 0x7f6168004c40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f617400c240 tx=0x7f61680057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f618482d640 1 -- 172.21.15.111:0/2504804299 reap_dead start 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f6186290640 1 -- 172.21.15.111:0/2504804299 shutdown_connections 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f6186290640 1 -- 172.21.15.111:0/2504804299 >> 172.21.15.111:0/2504804299 conn(0x7f61800b0cf0 msgr2=0x7f61800b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f6186290640 1 -- 172.21.15.111:0/2504804299 shutdown_connections 2023-12-04T05:39:58.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.571+0000 7f6186290640 1 -- 172.21.15.111:0/2504804299 wait complete. 2023-12-04T05:39:58.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.572+0000 7f6186290640 1 Processor -- start 2023-12-04T05:39:58.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.572+0000 7f6186290640 1 -- start start 2023-12-04T05:39:58.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f6186290640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61680047c0 0x7f61801519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f6186290640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6180152170 0x7f6180141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f6186290640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 0x7f6180154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f6186290640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6180151f30 con 0x7f618014daa0 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f6186290640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6180152730 con 0x7f6180152170 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f6186290640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618014cce0 con 0x7f61680047c0 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 0x7f6180154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 0x7f6180154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36518/0 (socket says 172.21.15.111:36518) 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 -- 172.21.15.111:0/1772417314 learned_addr learned my addr 172.21.15.111:0/1772417314 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f617f7fe640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6180152170 0x7f6180141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f617ffff640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61680047c0 0x7f61801519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 -- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61680047c0 msgr2=0x7f61801519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61680047c0 0x7f61801519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 -- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6180152170 msgr2=0x7f6180141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6180152170 0x7f6180141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.573+0000 7f618482d640 1 -- 172.21.15.111:0/1772417314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618015b620 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.574+0000 7f618482d640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 0x7f6180154460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61800aebf0 tx=0x7f61780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.574+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800e070 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.574+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178011ed0 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.574+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618014cfc0 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.574+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6178003990 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.574+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618014d580 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.575+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618014d7a0 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.575+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6178003b10 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.576+0000 7f617cff9640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f616c06b2a0 0x7f616c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.576+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f617808a370 con 0x7f618014daa0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.576+0000 7f617ffff640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f616c06b2a0 0x7f616c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.576+0000 7f617ffff640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f616c06b2a0 0x7f616c06d760 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6170005450 tx=0x7f6170009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.580+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6178055210 con 0x7f618014daa0 2023-12-04T05:39:58.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.744+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6180066f70 con 0x7f618014daa0 2023-12-04T05:39:58.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.744+0000 7f617cff9640 1 -- 172.21.15.111:0/1772417314 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6178058830 con 0x7f618014daa0 2023-12-04T05:39:58.746 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f616c06b2a0 msgr2=0x7f616c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f616c06b2a0 0x7f616c06d760 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6170005450 tx=0x7f6170009040 comp rx=0 tx=0).stop 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 msgr2=0x7f6180154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 0x7f6180154460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61800aebf0 tx=0x7f61780115a0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 shutdown_connections 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f616c06b2a0 0x7f616c06d760 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618014daa0 0x7f6180154460 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6180152170 0x7f6180141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 --2- 172.21.15.111:0/1772417314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61680047c0 0x7f61801519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 >> 172.21.15.111:0/1772417314 conn(0x7f61800b0cf0 msgr2=0x7f618015f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:58.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 shutdown_connections 2023-12-04T05:39:58.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.746+0000 7f6186290640 1 -- 172.21.15.111:0/1772417314 wait complete. 2023-12-04T05:39:58.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:39:58.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.861+0000 7f9280141640 1 Processor -- start 2023-12-04T05:39:58.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.861+0000 7f9280141640 1 -- start start 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f9280141640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f927815b0c0 0x7f927815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f9280141640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f9280141640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92780cd320 con 0x7f9278071b40 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f9280141640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92780ced80 con 0x7f927815b0c0 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f9280141640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92780cbee0 con 0x7f92780ce940 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f927d6dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f927d6dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:37268/0 (socket says 172.21.15.111:37268) 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f927d6dc640 1 -- 172.21.15.111:0/2753754535 learned_addr learned my addr 172.21.15.111:0/2753754535 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:58.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.862+0000 7f927dedd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f927815b0c0 0x7f927815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.863+0000 7f927d6dc640 1 -- 172.21.15.111:0/2753754535 >> v2:172.21.15.192:3300/0 conn(0x7f927815b0c0 msgr2=0x7f927815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.863+0000 7f927d6dc640 1 --2- 172.21.15.111:0/2753754535 >> v2:172.21.15.192:3300/0 conn(0x7f927815b0c0 0x7f927815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.863+0000 7f927d6dc640 1 -- 172.21.15.111:0/2753754535 >> v1:172.21.15.192:6789/0 conn(0x7f9278071b40 legacy=0x7f927815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.863+0000 7f927d6dc640 1 -- 172.21.15.111:0/2753754535 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780cc710 con 0x7f92780ce940 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f927d6dc640 1 --2- 172.21.15.111:0/2753754535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780cb9a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6191cd61d07338c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f927cedb640 1 -- 172.21.15.111:0/2753754535 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268031640 con 0x7f92780ce940 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f927cedb640 1 -- 172.21.15.111:0/2753754535 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92680317a0 con 0x7f92780ce940 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f927cedb640 1 -- 172.21.15.111:0/2753754535 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268031b30 con 0x7f92780ce940 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 -- 172.21.15.111:0/2753754535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 msgr2=0x7f92780cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 --2- 172.21.15.111:0/2753754535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780cb9a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926802b8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 -- 172.21.15.111:0/2753754535 shutdown_connections 2023-12-04T05:39:58.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 --2- 172.21.15.111:0/2753754535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780cb9a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 --2- 172.21.15.111:0/2753754535 >> v2:172.21.15.192:3300/0 conn(0x7f927815b0c0 0x7f927815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 -- 172.21.15.111:0/2753754535 >> 172.21.15.111:0/2753754535 conn(0x7f92780b0cf0 msgr2=0x7f92780b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:58.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 -- 172.21.15.111:0/2753754535 shutdown_connections 2023-12-04T05:39:58.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.864+0000 7f9280141640 1 -- 172.21.15.111:0/2753754535 wait complete. 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.865+0000 7f9280141640 1 Processor -- start 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 -- start start 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 0x7f9278145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927815b0c0 0x7f92780aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92780adfc0 con 0x7f9278071b40 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92780ad3b0 con 0x7f927815b0c0 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f9280141640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92780ad690 con 0x7f92780ce940 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f927dedd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 0x7f9278145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f927dedd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 0x7f9278145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36534/0 (socket says 172.21.15.111:36534) 2023-12-04T05:39:58.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f927dedd640 1 -- 172.21.15.111:0/2905653531 learned_addr learned my addr 172.21.15.111:0/2905653531 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f927e6de640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927815b0c0 0x7f92780aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.866+0000 7f927d6dc640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f927dedd640 1 -- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 msgr2=0x7f92780ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f927dedd640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f927dedd640 1 -- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927815b0c0 msgr2=0x7f92780aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f927dedd640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927815b0c0 0x7f92780aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f927dedd640 1 -- 172.21.15.111:0/2905653531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680094d0 con 0x7f9278071b40 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f927dedd640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 0x7f9278145ef0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f927400efc0 tx=0x7f927400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274009b80 con 0x7f9278071b40 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927400f040 con 0x7f9278071b40 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740146a0 con 0x7f9278071b40 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92780ad970 con 0x7f9278071b40 2023-12-04T05:39:58.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.867+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92780acef0 con 0x7f9278071b40 2023-12-04T05:39:58.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.868+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9278069360 con 0x7f9278071b40 2023-12-04T05:39:58.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.868+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9274014800 con 0x7f9278071b40 2023-12-04T05:39:58.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.869+0000 7f926e7fc640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925c06b2d0 0x7f925c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:58.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.869+0000 7f927d6dc640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925c06b2d0 0x7f925c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:58.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.869+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f927408a5e0 con 0x7f9278071b40 2023-12-04T05:39:58.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.869+0000 7f927d6dc640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925c06b2d0 0x7f925c06d790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9268036000 tx=0x7f926803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:58.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:58.873+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9274055480 con 0x7f9278071b40 2023-12-04T05:39:59.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.039+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92780a8710 con 0x7f9278071b40 2023-12-04T05:39:59.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.039+0000 7f926e7fc640 1 -- 172.21.15.111:0/2905653531 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9274058aa0 con 0x7f9278071b40 2023-12-04T05:39:59.041 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925c06b2d0 msgr2=0x7f925c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925c06b2d0 0x7f925c06d790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9268036000 tx=0x7f926803c040 comp rx=0 tx=0).stop 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 msgr2=0x7f9278145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 0x7f9278145ef0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f927400efc0 tx=0x7f927400c490 comp rx=0 tx=0).stop 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 shutdown_connections 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925c06b2d0 0x7f925c06d790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927815b0c0 0x7f92780aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92780ce940 0x7f92780ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 --2- 172.21.15.111:0/2905653531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9278071b40 0x7f9278145ef0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 >> 172.21.15.111:0/2905653531 conn(0x7f92780b0cf0 msgr2=0x7f927815cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 shutdown_connections 2023-12-04T05:39:59.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.041+0000 7f9280141640 1 -- 172.21.15.111:0/2905653531 wait complete. 2023-12-04T05:39:59.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:59.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.152+0000 7f45a2ce0640 1 Processor -- start 2023-12-04T05:39:59.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.152+0000 7f45a2ce0640 1 -- start start 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a2ce0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b20a0 0x7f459c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a2ce0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f459c0b2ad0 0x7f459c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a2ce0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459c0cdcb0 con 0x7f459c0d1780 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a2ce0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459c0d39e0 con 0x7f459c0b2ad0 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a2ce0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0d3cc0 con 0x7f459c0b20a0 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a127d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f459c0b2ad0 0x7f459c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a0a7c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f459c0d1780 0x7f459c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54914/0 (socket says 172.21.15.111:54914) 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.153+0000 7f45a0a7c640 1 -- 172.21.15.111:0/1339837964 learned_addr learned my addr 172.21.15.111:0/1339837964 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f459bfff640 1 --2- 172.21.15.111:0/1339837964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b20a0 0x7f459c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f45a127d640 1 -- 172.21.15.111:0/1339837964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b20a0 msgr2=0x7f459c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f45a127d640 1 --2- 172.21.15.111:0/1339837964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b20a0 0x7f459c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f45a127d640 1 -- 172.21.15.111:0/1339837964 >> v1:172.21.15.111:6789/0 conn(0x7f459c0d1780 legacy=0x7f459c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f45a127d640 1 -- 172.21.15.111:0/1339837964 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0cdf50 con 0x7f459c0b2ad0 2023-12-04T05:39:59.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f459b7fe640 1 -- 172.21.15.111:0/1339837964 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112065973 0 0) 0x7f459c0cdcb0 con 0x7f459c0d1780 2023-12-04T05:39:59.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f45a127d640 1 --2- 172.21.15.111:0/1339837964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b2ad0 0x7f459c0ce750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4594009d90 tx=0x7f459402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cf9382d07e0c83 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.154+0000 7f459b7fe640 1 -- 172.21.15.111:0/1339837964 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459402cbc0 con 0x7f459c0b2ad0 2023-12-04T05:39:59.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f459b7fe640 1 -- 172.21.15.111:0/1339837964 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459402cd20 con 0x7f459c0b2ad0 2023-12-04T05:39:59.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f459b7fe640 1 -- 172.21.15.111:0/1339837964 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594034660 con 0x7f459c0b2ad0 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1339837964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b2ad0 msgr2=0x7f459c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1339837964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b2ad0 0x7f459c0ce750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4594009d90 tx=0x7f459402be10 comp rx=0 tx=0).stop 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1339837964 shutdown_connections 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1339837964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b2ad0 0x7f459c0ce750 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1339837964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b20a0 0x7f459c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1339837964 >> 172.21.15.111:0/1339837964 conn(0x7f459c09c4e0 msgr2=0x7f459c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1339837964 shutdown_connections 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.155+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1339837964 wait complete. 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.156+0000 7f45a2ce0640 1 Processor -- start 2023-12-04T05:39:59.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.156+0000 7f45a2ce0640 1 -- start start 2023-12-04T05:39:59.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a2ce0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 0x7f459c14c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a2ce0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b2ad0 0x7f459c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a2ce0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f459c0d1780 0x7f459c0c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a2ce0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0cd280 con 0x7f459c0b20a0 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a2ce0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0cc670 con 0x7f459c0d1780 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a2ce0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c0cc950 con 0x7f459c0b2ad0 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 0x7f459c14c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 0x7f459c14c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36562/0 (socket says 172.21.15.111:36562) 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 -- 172.21.15.111:0/1978485507 learned_addr learned my addr 172.21.15.111:0/1978485507 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 -- 172.21.15.111:0/1978485507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b2ad0 msgr2=0x7f459c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b2ad0 0x7f459c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 -- 172.21.15.111:0/1978485507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f459c0d1780 msgr2=0x7f459c0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f459c0d1780 0x7f459c0c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.157+0000 7f45a0a7c640 1 -- 172.21.15.111:0/1978485507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594009a70 con 0x7f459c0b20a0 2023-12-04T05:39:59.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.158+0000 7f45a0a7c640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 0x7f459c14c0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c430 tx=0x7f458c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.158+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0074e0 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.158+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c007660 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.158+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c0ccc30 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.158+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c004a20 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.158+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c14c600 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.159+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c15d900 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.159+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f458c004b80 con 0x7f459c0b20a0 2023-12-04T05:39:59.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.159+0000 7f45997fa640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458806b2a0 0x7f458806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.160+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f458c089a60 con 0x7f459c0b20a0 2023-12-04T05:39:59.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.160+0000 7f459bfff640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458806b2a0 0x7f458806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.160+0000 7f459bfff640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458806b2a0 0x7f458806d760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f459c0d2240 tx=0x7f4590008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.164+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f458c054850 con 0x7f459c0b20a0 2023-12-04T05:39:59.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.367+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f459c0693d0 con 0x7f459c0b20a0 2023-12-04T05:39:59.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.368+0000 7f45997fa640 1 -- 172.21.15.111:0/1978485507 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f458c057e70 con 0x7f459c0b20a0 2023-12-04T05:39:59.369 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:59.370 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:59.370 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:59.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458806b2a0 msgr2=0x7f458806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458806b2a0 0x7f458806d760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f459c0d2240 tx=0x7f4590008040 comp rx=0 tx=0).stop 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 msgr2=0x7f459c14c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 0x7f459c14c0c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f458c00c430 tx=0x7f458c00c8f0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 shutdown_connections 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f458806b2a0 0x7f458806d760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f459c0d1780 0x7f459c0c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f459c0b2ad0 0x7f459c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 --2- 172.21.15.111:0/1978485507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f459c0b20a0 0x7f459c14c0c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 >> 172.21.15.111:0/1978485507 conn(0x7f459c09c4e0 msgr2=0x7f459c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 shutdown_connections 2023-12-04T05:39:59.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.370+0000 7f45a2ce0640 1 -- 172.21.15.111:0/1978485507 wait complete. 2023-12-04T05:39:59.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:39:59.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.480+0000 7fd1368c6640 1 Processor -- start 2023-12-04T05:39:59.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.480+0000 7fd1368c6640 1 -- start start 2023-12-04T05:39:59.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.481+0000 7fd1368c6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd1300b20a0 0x7fd1300d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.481+0000 7fd1368c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b2ad0 0x7fd1300ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.481+0000 7fd1368c6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1300cdcb0 con 0x7fd1300d1780 2023-12-04T05:39:59.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.481+0000 7fd1368c6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1300d39e0 con 0x7fd1300b20a0 2023-12-04T05:39:59.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.481+0000 7fd1368c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300d3cc0 con 0x7fd1300b2ad0 2023-12-04T05:39:59.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12f7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd1300b20a0 0x7fd1300d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd134e63640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b2ad0 0x7fd1300ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12ffff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd1300d1780 0x7fd1300d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50924/0 (socket says 172.21.15.111:50924) 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12f7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd1300b20a0 0x7fd1300d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:47020/0 (socket says 172.21.15.111:47020) 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12ffff640 1 -- 172.21.15.111:0/387497084 learned_addr learned my addr 172.21.15.111:0/387497084 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12f7fe640 1 -- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b2ad0 msgr2=0x7fd1300ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12f7fe640 1 --2- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b2ad0 0x7fd1300ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12f7fe640 1 -- 172.21.15.111:0/387497084 >> v1:172.21.15.192:6789/0 conn(0x7fd1300d1780 legacy=0x7fd1300d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12f7fe640 1 -- 172.21.15.111:0/387497084 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1300cdf50 con 0x7fd1300b20a0 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd12effd640 1 -- 172.21.15.111:0/387497084 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422359824 0 0) 0x7fd1300cdcb0 con 0x7fd1300d1780 2023-12-04T05:39:59.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.482+0000 7fd134e63640 1 --2- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b2ad0 0x7fd1300ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:39:59.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd12f7fe640 1 --2- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b20a0 0x7fd1300d01e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1240097a0 tx=0x7fd12402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797870ed3e9bf1ff server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd12effd640 1 -- 172.21.15.111:0/387497084 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12402e070 con 0x7fd1300b20a0 2023-12-04T05:39:59.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd12effd640 1 -- 172.21.15.111:0/387497084 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1240315a0 con 0x7fd1300b20a0 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd12effd640 1 -- 172.21.15.111:0/387497084 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124031930 con 0x7fd1300b20a0 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 -- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b20a0 msgr2=0x7fd1300d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 --2- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b20a0 0x7fd1300d01e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1240097a0 tx=0x7fd12402b840 comp rx=0 tx=0).stop 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 -- 172.21.15.111:0/387497084 shutdown_connections 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 --2- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b2ad0 0x7fd1300ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 --2- 172.21.15.111:0/387497084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b20a0 0x7fd1300d01e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 -- 172.21.15.111:0/387497084 >> 172.21.15.111:0/387497084 conn(0x7fd13009c4e0 msgr2=0x7fd1300d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.483+0000 7fd1368c6640 1 -- 172.21.15.111:0/387497084 shutdown_connections 2023-12-04T05:39:59.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.484+0000 7fd1368c6640 1 -- 172.21.15.111:0/387497084 wait complete. 2023-12-04T05:39:59.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 Processor -- start 2023-12-04T05:39:59.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 -- start start 2023-12-04T05:39:59.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b20a0 0x7fd1301578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b2ad0 0x7fd1300ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 0x7fd1300ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130157e30 con 0x7fd1300d1780 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130158e20 con 0x7fd1300b2ad0 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.485+0000 7fd1368c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd130159100 con 0x7fd1300b20a0 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 0x7fd1300ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 0x7fd1300ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36566/0 (socket says 172.21.15.111:36566) 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 -- 172.21.15.111:0/1061383521 learned_addr learned my addr 172.21.15.111:0/1061383521 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd12f7fe640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b2ad0 0x7fd1300ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd12ffff640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b20a0 0x7fd1301578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 -- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b20a0 msgr2=0x7fd1301578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b20a0 0x7fd1301578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 -- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b2ad0 msgr2=0x7fd1300ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b2ad0 0x7fd1300ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 -- 172.21.15.111:0/1061383521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd128009a70 con 0x7fd1300d1780 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd12f7fe640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b2ad0 0x7fd1300ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:39:59.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd134e63640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 0x7fd1300ca7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd12800efc0 tx=0x7fd128009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128009d70 con 0x7fd1300d1780 2023-12-04T05:39:59.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800f040 con 0x7fd1300d1780 2023-12-04T05:39:59.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd124009400 con 0x7fd1300d1780 2023-12-04T05:39:59.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128003990 con 0x7fd1300d1780 2023-12-04T05:39:59.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.486+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1301596c0 con 0x7fd1300d1780 2023-12-04T05:39:59.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.487+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd130147950 con 0x7fd1300d1780 2023-12-04T05:39:59.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.488+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd1280144a0 con 0x7fd1300d1780 2023-12-04T05:39:59.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.488+0000 7fd12cff9640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd11c06b2d0 0x7fd11c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.488+0000 7fd12ffff640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd11c06b2d0 0x7fd11c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.488+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd12808a880 con 0x7fd1300d1780 2023-12-04T05:39:59.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.489+0000 7fd12ffff640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd11c06b2d0 0x7fd11c06d790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd120000f60 tx=0x7fd120009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.493+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd130147950 con 0x7fd1300d1780 2023-12-04T05:39:59.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.658+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd130147950 con 0x7fd1300d1780 2023-12-04T05:39:59.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.659+0000 7fd12cff9640 1 -- 172.21.15.111:0/1061383521 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16674 (secure 0 0 0) 0x7fd130147950 con 0x7fd1300d1780 2023-12-04T05:39:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:59.661 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":22,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:58.336257+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd11c06b2d0 msgr2=0x7fd11c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd11c06b2d0 0x7fd11c06d790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd120000f60 tx=0x7fd120009040 comp rx=0 tx=0).stop 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 msgr2=0x7fd1300ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 0x7fd1300ca7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd12800efc0 tx=0x7fd128009f90 comp rx=0 tx=0).stop 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 shutdown_connections 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd11c06b2d0 0x7fd11c06d790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1300d1780 0x7fd1300ca7b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1300b2ad0 0x7fd1300ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 --2- 172.21.15.111:0/1061383521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1300b20a0 0x7fd1301578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 >> 172.21.15.111:0/1061383521 conn(0x7fd13009c4e0 msgr2=0x7fd13006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 shutdown_connections 2023-12-04T05:39:59.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.661+0000 7fd1368c6640 1 -- 172.21.15.111:0/1061383521 wait complete. 2023-12-04T05:39:59.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-04T05:39:59.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:39:59.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.772+0000 7f5d560da640 1 Processor -- start 2023-12-04T05:39:59.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.772+0000 7f5d560da640 1 -- start start 2023-12-04T05:39:59.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d560da640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5d500d01e0 0x7f5d500ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d560da640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500cdcb0 con 0x7f5d500b20a0 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d560da640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500d39e0 con 0x7f5d500d1780 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d560da640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500d05f0 con 0x7f5d500d01e0 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d47fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5d500b20a0 0x7f5d500b2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54924/0 (socket says 172.21.15.111:54924) 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d47fff640 1 -- 172.21.15.111:0/1989159437 learned_addr learned my addr 172.21.15.111:0/1989159437 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503824383 0 0) 0x7f5d500cdcb0 con 0x7f5d500b20a0 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d3c003520 con 0x7f5d500b20a0 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4ffff640 1 --2- 172.21.15.111:0/1989159437 >> v2:172.21.15.192:3300/0 conn(0x7f5d500d01e0 0x7f5d500ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1912194789 0 0) 0x7f5d3c003520 con 0x7f5d500b20a0 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 >> v2:172.21.15.192:3300/0 conn(0x7f5d500d01e0 msgr2=0x7f5d500ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 --2- 172.21.15.111:0/1989159437 >> v2:172.21.15.192:3300/0 conn(0x7f5d500d01e0 0x7f5d500ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 >> v1:172.21.15.192:6789/0 conn(0x7f5d500d1780 legacy=0x7f5d500d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500d3cc0 con 0x7f5d500b20a0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f5d38001670 con 0x7f5d500b20a0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 >> v1:172.21.15.111:6789/0 conn(0x7f5d500b20a0 legacy=0x7f5d500b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c003970 0x7f5d3c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d3c0047c0 0x7f5d3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c0052a0 0x7f5d3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500d05f0 con 0x7f5d3c003970 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c00c150 con 0x7f5d3c0052a0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c00c310 con 0x7f5d3c0047c0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.773+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c003970 0x7f5d3c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d47fff640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d3c0047c0 0x7f5d3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4ffff640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c0052a0 0x7f5d3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4f7fe640 1 -- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d3c0047c0 msgr2=0x7f5d3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d3c0047c0 0x7f5d3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4f7fe640 1 -- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c0052a0 msgr2=0x7f5d3c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c0052a0 0x7f5d3c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4ffff640 1 -- 172.21.15.111:0/1989159437 reap_dead start 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4f7fe640 1 -- 172.21.15.111:0/1989159437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c003520 con 0x7f5d3c003970 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c003970 0x7f5d3c003d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5d40003d80 tx=0x7f5d3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c473db3212b0f0c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40032600 con 0x7f5d3c003970 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4002e050 con 0x7f5d3c003970 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.774+0000 7f5d4effd640 1 -- 172.21.15.111:0/1989159437 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400329e0 con 0x7f5d3c003970 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 -- 172.21.15.111:0/1989159437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c003970 msgr2=0x7f5d3c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c003970 0x7f5d3c003d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5d40003d80 tx=0x7f5d3c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 -- 172.21.15.111:0/1989159437 shutdown_connections 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 --2- 172.21.15.111:0/1989159437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c003970 0x7f5d3c003d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 -- 172.21.15.111:0/1989159437 >> 172.21.15.111:0/1989159437 conn(0x7f5d5009c4e0 msgr2=0x7f5d500d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 -- 172.21.15.111:0/1989159437 shutdown_connections 2023-12-04T05:39:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.775+0000 7f5d560da640 1 -- 172.21.15.111:0/1989159437 wait complete. 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.776+0000 7f5d560da640 1 Processor -- start 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.776+0000 7f5d560da640 1 -- start start 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d560da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c003970 0x7f5d501588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d560da640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 0x7f5d501614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d560da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d500b71c0 0x7f5d500b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d560da640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b8b90 con 0x7f5d50161060 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d560da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50161a20 con 0x7f5d3c003970 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d560da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b7f10 con 0x7f5d500b71c0 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 0x7f5d501614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 0x7f5d501614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36580/0 (socket says 172.21.15.111:36580) 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 -- 172.21.15.111:0/3727221326 learned_addr learned my addr 172.21.15.111:0/3727221326 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c003970 0x7f5d501588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d4ffff640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d500b71c0 0x7f5d500b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 -- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d500b71c0 msgr2=0x7f5d500b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d500b71c0 0x7f5d500b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 -- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c003970 msgr2=0x7f5d501588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c003970 0x7f5d501588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.777+0000 7f5d47fff640 1 -- 172.21.15.111:0/3727221326 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500b81f0 con 0x7f5d50161060 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.778+0000 7f5d47fff640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 0x7f5d501614e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e7b0 tx=0x7f5d3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.778+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800c8d0 con 0x7f5d50161060 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.778+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800ca50 con 0x7f5d50161060 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.778+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d500b84d0 con 0x7f5d50161060 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.778+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800ce00 con 0x7f5d50161060 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.778+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500b7b80 con 0x7f5d50161060 2023-12-04T05:39:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.779+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d500693d0 con 0x7f5d50161060 2023-12-04T05:39:59.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.779+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5d38013420 con 0x7f5d50161060 2023-12-04T05:39:59.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.779+0000 7f5d4cff9640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2d0 0x7f5d3006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:39:59.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.780+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5d38089920 con 0x7f5d50161060 2023-12-04T05:39:59.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.780+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2d0 0x7f5d3006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:39:59.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.780+0000 7f5d4f7fe640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2d0 0x7f5d3006d790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d40034000 tx=0x7f5d40009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:39:59.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.784+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d38054710 con 0x7f5d50161060 2023-12-04T05:39:59.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.984+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d500d2670 con 0x7f5d50161060 2023-12-04T05:39:59.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.984+0000 7f5d4cff9640 1 -- 172.21.15.111:0/3727221326 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f5d38057d30 con 0x7f5d50161060 2023-12-04T05:39:59.986 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:39:59.986 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:39:59.986 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2d0 msgr2=0x7f5d3006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2d0 0x7f5d3006d790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d40034000 tx=0x7f5d40009030 comp rx=0 tx=0).stop 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 msgr2=0x7f5d501614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 0x7f5d501614e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e7b0 tx=0x7f5d3800ec70 comp rx=0 tx=0).stop 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 shutdown_connections 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5d3006b2d0 0x7f5d3006d790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d500b71c0 0x7f5d500b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d50161060 0x7f5d501614e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 --2- 172.21.15.111:0/3727221326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d3c003970 0x7f5d501588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:39:59.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.986+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 >> 172.21.15.111:0/3727221326 conn(0x7f5d5009c4e0 msgr2=0x7f5d500ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:39:59.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.987+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 shutdown_connections 2023-12-04T05:39:59.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:39:59.987+0000 7f5d560da640 1 -- 172.21.15.111:0/3727221326 wait complete. 2023-12-04T05:39:59.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:00.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.098+0000 7f5153168640 1 Processor -- start 2023-12-04T05:40:00.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.098+0000 7f5153168640 1 -- start start 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5153168640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5153168640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f514c0b2ad0 0x7f514c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5153168640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c0cdcb0 con 0x7f514c0b20a0 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5153168640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c0d39e0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5153168640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0d3cc0 con 0x7f514c0d1780 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5151705640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f514c0b2ad0 0x7f514c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5151705640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f514c0b2ad0 0x7f514c0ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36590/0 (socket says 172.21.15.111:36590) 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.099+0000 7f5151705640 1 -- 172.21.15.111:0/1404324295 learned_addr learned my addr 172.21.15.111:0/1404324295 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5150f04640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5151705640 1 -- 172.21.15.111:0/1404324295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 msgr2=0x7f514c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5151705640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5151705640 1 -- 172.21.15.111:0/1404324295 >> v1:172.21.15.192:6789/0 conn(0x7f514c0b20a0 legacy=0x7f514c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5151705640 1 -- 172.21.15.111:0/1404324295 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0cdf50 con 0x7f514c0b2ad0 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5150f04640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.100+0000 7f5151705640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ce750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5144009870 tx=0x7f514402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71391a24f30b765 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f514b7fe640 1 -- 172.21.15.111:0/1404324295 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440317c0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f514b7fe640 1 -- 172.21.15.111:0/1404324295 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144031920 con 0x7f514c0b2ad0 2023-12-04T05:40:00.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f514b7fe640 1 -- 172.21.15.111:0/1404324295 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144031cb0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 -- 172.21.15.111:0/1404324295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 msgr2=0x7f514c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ce750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5144009870 tx=0x7f514402b860 comp rx=0 tx=0).stop 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 -- 172.21.15.111:0/1404324295 shutdown_connections 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ce750 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 --2- 172.21.15.111:0/1404324295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 -- 172.21.15.111:0/1404324295 >> 172.21.15.111:0/1404324295 conn(0x7f514c09c4e0 msgr2=0x7f514c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 -- 172.21.15.111:0/1404324295 shutdown_connections 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.101+0000 7f5153168640 1 -- 172.21.15.111:0/1404324295 wait complete. 2023-12-04T05:40:00.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.102+0000 7f5153168640 1 Processor -- start 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.102+0000 7f5153168640 1 -- start start 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5153168640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c0b20a0 0x7f514c0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5153168640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5153168640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5153168640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1f40b0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5153168640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0cb250 con 0x7f514c0b20a0 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5153168640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0cb4f0 con 0x7f514c0d1780 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f514bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f514bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36602/0 (socket says 172.21.15.111:36602) 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f514bfff640 1 -- 172.21.15.111:0/3889765867 learned_addr learned my addr 172.21.15.111:0/3889765867 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:00.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5150f04640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c0b20a0 0x7f514c0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.103+0000 7f5151705640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f514bfff640 1 -- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 msgr2=0x7f514c1f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f514bfff640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c1f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f514bfff640 1 -- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c0b20a0 msgr2=0x7f514c0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f514bfff640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c0b20a0 0x7f514c0cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f514bfff640 1 -- 172.21.15.111:0/3889765867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440094d0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f5151705640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f514bfff640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ccc70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c7c0 tx=0x7f513c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c00aa50 con 0x7f514c0b2ad0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c00abb0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0cb7d0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c012420 con 0x7f514c0b2ad0 2023-12-04T05:40:00.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.104+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c0cd370 con 0x7f514c0b2ad0 2023-12-04T05:40:00.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.105+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c0cc510 con 0x7f514c0b2ad0 2023-12-04T05:40:00.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.105+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f513c012580 con 0x7f514c0b2ad0 2023-12-04T05:40:00.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.106+0000 7f51497fa640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f512c06b2a0 0x7f512c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.106+0000 7f5150f04640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f512c06b2a0 0x7f512c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.106+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f513c089b10 con 0x7f514c0b2ad0 2023-12-04T05:40:00.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.106+0000 7f5150f04640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f512c06b2a0 0x7f512c06d760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f513400aac0 tx=0x7f5134008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.110+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f513c054900 con 0x7f514c0b2ad0 2023-12-04T05:40:00.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.309+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f514c0693d0 con 0x7f514c0b2ad0 2023-12-04T05:40:00.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.310+0000 7f51497fa640 1 -- 172.21.15.111:0/3889765867 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f513c016090 con 0x7f514c0b2ad0 2023-12-04T05:40:00.311 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:00.312 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2279652107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2279652107},{"type":"v1","addr":"172.21.15.111:6837","nonce":2279652107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2934281532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2934281532},{"type":"v1","addr":"172.21.15.192:6834","nonce":2934281532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1077798903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1077798903},{"type":"v1","addr":"172.21.15.192:6835","nonce":1077798903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:39:16.542901+0000","modified":"2023-12-04T05:39:18.555593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6836/4088659597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4088659597},{"type":"v1","addr":"172.21.15.111:6836","nonce":4088659597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-04T05:40:00.312 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2023-12-04T05:40:00.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f512c06b2a0 msgr2=0x7f512c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f512c06b2a0 0x7f512c06d760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f513400aac0 tx=0x7f5134008040 comp rx=0 tx=0).stop 2023-12-04T05:40:00.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 msgr2=0x7f514c0ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ccc70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c7c0 tx=0x7f513c00cc80 comp rx=0 tx=0).stop 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 shutdown_connections 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f512c06b2a0 0x7f512c06d760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c0d1780 0x7f514c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f514c0b2ad0 0x7f514c0ccc70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 --2- 172.21.15.111:0/3889765867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c0b20a0 0x7f514c0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 >> 172.21.15.111:0/3889765867 conn(0x7f514c09c4e0 msgr2=0x7f514c0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 shutdown_connections 2023-12-04T05:40:00.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.312+0000 7f5153168640 1 -- 172.21.15.111:0/3889765867 wait complete. 2023-12-04T05:40:00.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:00.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.423+0000 7eff3b82b640 1 Processor -- start 2023-12-04T05:40:00.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.423+0000 7eff3b82b640 1 -- start start 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff3b82b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7eff341460c0 0x7eff340b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff3b82b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff341464a0 0x7eff340d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff3b82b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff34161d10 con 0x7eff340d15f0 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff3b82b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff34146fa0 con 0x7eff341460c0 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff3b82b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3415d990 con 0x7eff341464a0 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff395c7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7eff341460c0 0x7eff340b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff39dc8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7eff340d15f0 0x7eff340b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47216/0 (socket says 172.21.15.111:47216) 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff39dc8640 1 -- 172.21.15.111:0/4186726710 learned_addr learned my addr 172.21.15.111:0/4186726710 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.424+0000 7eff38dc6640 1 --2- 172.21.15.111:0/4186726710 >> v2:172.21.15.192:3300/0 conn(0x7eff341464a0 0x7eff340d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff395c7640 1 -- 172.21.15.111:0/4186726710 >> v2:172.21.15.192:3300/0 conn(0x7eff341464a0 msgr2=0x7eff340d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:00.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff395c7640 1 --2- 172.21.15.111:0/4186726710 >> v2:172.21.15.192:3300/0 conn(0x7eff341464a0 0x7eff340d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff395c7640 1 -- 172.21.15.111:0/4186726710 >> v1:172.21.15.111:6789/0 conn(0x7eff340d15f0 legacy=0x7eff340b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff395c7640 1 -- 172.21.15.111:0/4186726710 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34161100 con 0x7eff341460c0 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff33fff640 1 -- 172.21.15.111:0/4186726710 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112770142 0 0) 0x7eff34161d10 con 0x7eff340d15f0 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff38dc6640 1 --2- 172.21.15.111:0/4186726710 >> v2:172.21.15.192:3300/0 conn(0x7eff341464a0 0x7eff340d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff395c7640 1 --2- 172.21.15.111:0/4186726710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff341460c0 0x7eff340b4a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff3406d740 tx=0x7eff2c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddfc36183aab529 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff33fff640 1 -- 172.21.15.111:0/4186726710 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c02cc60 con 0x7eff341460c0 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.425+0000 7eff33fff640 1 -- 172.21.15.111:0/4186726710 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c02cdc0 con 0x7eff341460c0 2023-12-04T05:40:00.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff33fff640 1 -- 172.21.15.111:0/4186726710 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c034660 con 0x7eff341460c0 2023-12-04T05:40:00.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 -- 172.21.15.111:0/4186726710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff341460c0 msgr2=0x7eff340b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 --2- 172.21.15.111:0/4186726710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff341460c0 0x7eff340b4a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff3406d740 tx=0x7eff2c02beb0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 -- 172.21.15.111:0/4186726710 shutdown_connections 2023-12-04T05:40:00.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 --2- 172.21.15.111:0/4186726710 >> v2:172.21.15.192:3300/0 conn(0x7eff341464a0 0x7eff340d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 --2- 172.21.15.111:0/4186726710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff341460c0 0x7eff340b4a00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 -- 172.21.15.111:0/4186726710 >> 172.21.15.111:0/4186726710 conn(0x7eff3406a4c0 msgr2=0x7eff3406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 -- 172.21.15.111:0/4186726710 shutdown_connections 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.426+0000 7eff3b82b640 1 -- 172.21.15.111:0/4186726710 wait complete. 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.427+0000 7eff3b82b640 1 Processor -- start 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.427+0000 7eff3b82b640 1 -- start start 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff3b82b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 0x7eff34140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff3b82b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff341460c0 0x7eff34140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff3b82b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341464a0 0x7eff3414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff3b82b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3414df20 con 0x7eff340d15f0 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff3b82b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3414d310 con 0x7eff341460c0 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff3b82b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3414d5f0 con 0x7eff341464a0 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 0x7eff34140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 0x7eff34140440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40898/0 (socket says 172.21.15.111:40898) 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 -- 172.21.15.111:0/2974877920 learned_addr learned my addr 172.21.15.111:0/2974877920 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff38dc6640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff341460c0 0x7eff34140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff39dc8640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341464a0 0x7eff3414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 -- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341464a0 msgr2=0x7eff3414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341464a0 0x7eff3414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 -- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff341460c0 msgr2=0x7eff34140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff341460c0 0x7eff34140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff395c7640 1 -- 172.21.15.111:0/2974877920 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c009a70 con 0x7eff340d15f0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.428+0000 7eff38dc6640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff341460c0 0x7eff34140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.429+0000 7eff395c7640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 0x7eff34140440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7eff2c037040 tx=0x7eff2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.429+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c040650 con 0x7eff340d15f0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.429+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c0407b0 con 0x7eff340d15f0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.429+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3414d8d0 con 0x7eff340d15f0 2023-12-04T05:40:00.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.429+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c040b60 con 0x7eff340d15f0 2023-12-04T05:40:00.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.429+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff341525c0 con 0x7eff340d15f0 2023-12-04T05:40:00.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.430+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff340693d0 con 0x7eff340d15f0 2023-12-04T05:40:00.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.430+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7eff2c040cc0 con 0x7eff340d15f0 2023-12-04T05:40:00.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.431+0000 7eff31ffb640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff2006b2a0 0x7eff2006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.431+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7eff2c0acaf0 con 0x7eff340d15f0 2023-12-04T05:40:00.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.431+0000 7eff38dc6640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff2006b2a0 0x7eff2006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.431+0000 7eff38dc6640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff2006b2a0 0x7eff2006d760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7eff340b5440 tx=0x7eff28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.436 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.435+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff2c0778e0 con 0x7eff340d15f0 2023-12-04T05:40:00.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.604+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff340d0c50 con 0x7eff340d15f0 2023-12-04T05:40:00.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.605+0000 7eff31ffb640 1 -- 172.21.15.111:0/2974877920 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16674 (secure 0 0 0) 0x7eff2c07af00 con 0x7eff340d15f0 2023-12-04T05:40:00.606 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:00.607 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":22,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:39:58.336257+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:39:14.548588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-04T05:39:15.521726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:00.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff2006b2a0 msgr2=0x7eff2006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff2006b2a0 0x7eff2006d760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7eff340b5440 tx=0x7eff28008040 comp rx=0 tx=0).stop 2023-12-04T05:40:00.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 msgr2=0x7eff34140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 0x7eff34140440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7eff2c037040 tx=0x7eff2c004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 shutdown_connections 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7eff2006b2a0 0x7eff2006d760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341464a0 0x7eff3414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff341460c0 0x7eff34140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 --2- 172.21.15.111:0/2974877920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff340d15f0 0x7eff34140440 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 >> 172.21.15.111:0/2974877920 conn(0x7eff3406a4c0 msgr2=0x7eff3406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 shutdown_connections 2023-12-04T05:40:00.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.607+0000 7eff3b82b640 1 -- 172.21.15.111:0/2974877920 wait complete. 2023-12-04T05:40:00.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-04T05:40:00.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.717+0000 7f8b3b4c2640 1 Processor -- start 2023-12-04T05:40:00.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 -- start start 2023-12-04T05:40:00.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b3413cee0 0x7f8b3406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8b34140200 0x7f8b3406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3415a490 0x7f8b3415a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340aea20 con 0x7f8b3413cee0 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3413d2c0 con 0x7f8b34140200 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.718+0000 7f8b3b4c2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3415b3a0 con 0x7f8b3415a490 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b3413cee0 0x7f8b3406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8b3413cee0 0x7f8b3406c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40908/0 (socket says 172.21.15.111:40908) 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 -- 172.21.15.111:0/131255924 learned_addr learned my addr 172.21.15.111:0/131255924 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b38a5d640 1 --2- 172.21.15.111:0/131255924 >> v2:172.21.15.192:3300/0 conn(0x7f8b34140200 0x7f8b3406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b39a5f640 1 --2- 172.21.15.111:0/131255924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3415a490 0x7f8b3415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 -- 172.21.15.111:0/131255924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3415a490 msgr2=0x7f8b3415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 --2- 172.21.15.111:0/131255924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3415a490 0x7f8b3415a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 -- 172.21.15.111:0/131255924 >> v2:172.21.15.192:3300/0 conn(0x7f8b34140200 msgr2=0x7f8b3406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 --2- 172.21.15.111:0/131255924 >> v2:172.21.15.192:3300/0 conn(0x7f8b34140200 0x7f8b3406c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.719+0000 7f8b3925e640 1 -- 172.21.15.111:0/131255924 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3415ae60 con 0x7f8b3413cee0 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3925e640 1 --2- 172.21.15.111:0/131255924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3413cee0 0x7f8b3406c250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b34067f50 tx=0x7f8b3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539990bfbf7a87b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b2bfff640 1 -- 172.21.15.111:0/131255924 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b300317e0 con 0x7f8b3413cee0 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b2bfff640 1 -- 172.21.15.111:0/131255924 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30031940 con 0x7f8b3413cee0 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b2bfff640 1 -- 172.21.15.111:0/131255924 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30031cd0 con 0x7f8b3413cee0 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/131255924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3413cee0 msgr2=0x7f8b3406c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/131255924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3413cee0 0x7f8b3406c250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b34067f50 tx=0x7f8b3002b880 comp rx=0 tx=0).stop 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/131255924 shutdown_connections 2023-12-04T05:40:00.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/131255924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3415a490 0x7f8b3415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/131255924 >> v2:172.21.15.192:3300/0 conn(0x7f8b34140200 0x7f8b3406c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/131255924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3413cee0 0x7f8b3406c250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/131255924 >> 172.21.15.111:0/131255924 conn(0x7f8b3406a4c0 msgr2=0x7f8b3406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/131255924 shutdown_connections 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.720+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/131255924 wait complete. 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 Processor -- start 2023-12-04T05:40:00.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 -- start start 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3413cee0 0x7f8b34150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34140200 0x7f8b34156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 0x7f8b341572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b4510 con 0x7f8b3415a490 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3413bfd0 con 0x7f8b34140200 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b3b4c2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34150650 con 0x7f8b3413cee0 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b39a5f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 0x7f8b341572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b39a5f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 0x7f8b341572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40920/0 (socket says 172.21.15.111:40920) 2023-12-04T05:40:00.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.722+0000 7f8b39a5f640 1 -- 172.21.15.111:0/4221953248 learned_addr learned my addr 172.21.15.111:0/4221953248 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b38a5d640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34140200 0x7f8b34156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b3925e640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3413cee0 0x7f8b34150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b39a5f640 1 -- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3413cee0 msgr2=0x7f8b34150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b39a5f640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3413cee0 0x7f8b34150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b39a5f640 1 -- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34140200 msgr2=0x7f8b34156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b39a5f640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34140200 0x7f8b34156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b39a5f640 1 -- 172.21.15.111:0/4221953248 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300094d0 con 0x7f8b3415a490 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b38a5d640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34140200 0x7f8b34156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b39a5f640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 0x7f8b341572a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009d30 tx=0x7f8b2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c009ba0 con 0x7f8b3415a490 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00f040 con 0x7f8b3415a490 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b34152960 con 0x7f8b3415a490 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0146c0 con 0x7f8b3415a490 2023-12-04T05:40:00.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.723+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340b4230 con 0x7f8b3415a490 2023-12-04T05:40:00.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.724+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b340693d0 con 0x7f8b3415a490 2023-12-04T05:40:00.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.725+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8b2c014820 con 0x7f8b3415a490 2023-12-04T05:40:00.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.725+0000 7f8b29ffb640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8b1806b2a0 0x7f8b1806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:00.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.725+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8b2c08a610 con 0x7f8b3415a490 2023-12-04T05:40:00.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.725+0000 7f8b3925e640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8b1806b2a0 0x7f8b1806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:00.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.725+0000 7f8b3925e640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8b1806b2a0 0x7f8b1806d760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b30036000 tx=0x7f8b3003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:00.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.729+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b2c055490 con 0x7f8b3415a490 2023-12-04T05:40:00.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.930+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8b3406d170 con 0x7f8b3415a490 2023-12-04T05:40:00.931 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-12-04T05:40:00.930+0000 7fb27d104640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-04T05:40:00.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.932+0000 7f8b29ffb640 1 -- 172.21.15.111:0/4221953248 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8b2c058ab0 con 0x7f8b3415a490 2023-12-04T05:40:00.933 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-04T05:40:00.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8b1806b2a0 msgr2=0x7f8b1806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8b1806b2a0 0x7f8b1806d760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b30036000 tx=0x7f8b3003d040 comp rx=0 tx=0).stop 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 msgr2=0x7f8b341572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 0x7f8b341572a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009d30 tx=0x7f8b2c00c490 comp rx=0 tx=0).stop 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 shutdown_connections 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f8b1806b2a0 0x7f8b1806d760 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b3415a490 0x7f8b341572a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34140200 0x7f8b34156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 --2- 172.21.15.111:0/4221953248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b3413cee0 0x7f8b34150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 >> 172.21.15.111:0/4221953248 conn(0x7f8b3406a4c0 msgr2=0x7f8b340ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 shutdown_connections 2023-12-04T05:40:00.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:00.934+0000 7f8b3b4c2640 1 -- 172.21.15.111:0/4221953248 wait complete. 2023-12-04T05:40:00.937 INFO:tasks.ceph.mds.a.smithi111.stderr: -2634> 2023-12-04T05:39:20.203+0000 7f3082083640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-04T05:40:00.947 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-04T05:40:00.965 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2023-12-04T05:40:01.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.048+0000 7f2ac951d640 1 Processor -- start 2023-12-04T05:40:01.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.048+0000 7f2ac951d640 1 -- start start 2023-12-04T05:40:01.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac951d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2ac415b480 0x7f2ac40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac951d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac951d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ac415edd0 0x7f2ac415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac951d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac415f780 con 0x7f2ac415b480 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac951d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac40cbca0 con 0x7f2ac415edd0 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac951d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40cbf40 con 0x7f2ac40cf9f0 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2ac415b480 0x7f2ac40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2ac415b480 0x7f2ac40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40956/0 (socket says 172.21.15.111:40956) 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 -- 172.21.15.111:0/647427373 learned_addr learned my addr 172.21.15.111:0/647427373 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 -- 172.21.15.111:0/647427373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 msgr2=0x7f2ac40cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/647427373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 -- 172.21.15.111:0/647427373 >> v2:172.21.15.192:3300/0 conn(0x7f2ac415edd0 msgr2=0x7f2ac415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:01.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.049+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/647427373 >> v2:172.21.15.192:3300/0 conn(0x7f2ac415edd0 0x7f2ac415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.050+0000 7f2ac2ffd640 1 -- 172.21.15.111:0/647427373 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40cc220 con 0x7f2ac415b480 2023-12-04T05:40:01.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.050+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/647427373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415b480 0x7f2ac40b3590 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4071e60 tx=0x7f2ab402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3387da856a733d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:01.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.050+0000 7f2ac1ffb640 1 -- 172.21.15.111:0/647427373 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402ca70 con 0x7f2ac415b480 2023-12-04T05:40:01.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.050+0000 7f2ac1ffb640 1 -- 172.21.15.111:0/647427373 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab402cbd0 con 0x7f2ac415b480 2023-12-04T05:40:01.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.050+0000 7f2ac1ffb640 1 -- 172.21.15.111:0/647427373 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40343f0 con 0x7f2ac415b480 2023-12-04T05:40:01.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 -- 172.21.15.111:0/647427373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415b480 msgr2=0x7f2ac40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:01.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 --2- 172.21.15.111:0/647427373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415b480 0x7f2ac40b3590 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4071e60 tx=0x7f2ab402bed0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 -- 172.21.15.111:0/647427373 shutdown_connections 2023-12-04T05:40:01.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 --2- 172.21.15.111:0/647427373 >> v2:172.21.15.192:3300/0 conn(0x7f2ac415edd0 0x7f2ac415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 --2- 172.21.15.111:0/647427373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 --2- 172.21.15.111:0/647427373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415b480 0x7f2ac40b3590 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 -- 172.21.15.111:0/647427373 >> 172.21.15.111:0/647427373 conn(0x7f2ac40b2a70 msgr2=0x7f2ac40b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:01.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 -- 172.21.15.111:0/647427373 shutdown_connections 2023-12-04T05:40:01.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.051+0000 7f2ac951d640 1 -- 172.21.15.111:0/647427373 wait complete. 2023-12-04T05:40:01.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.052+0000 7f2ac951d640 1 Processor -- start 2023-12-04T05:40:01.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.052+0000 7f2ac951d640 1 -- start start 2023-12-04T05:40:01.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac951d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac951d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ac415b480 0x7f2ac40b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac951d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 0x7f2ac40af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac951d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40c94c0 con 0x7f2ac415edd0 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac951d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40b6c90 con 0x7f2ac415b480 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac951d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40c88b0 con 0x7f2ac40cf9f0 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 0x7f2ac40af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 0x7f2ac40af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40962/0 (socket says 172.21.15.111:40962) 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 -- 172.21.15.111:0/991902113 learned_addr learned my addr 172.21.15.111:0/991902113 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 -- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 msgr2=0x7f2ac40c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac27fc640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ac415b480 0x7f2ac40b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:01.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 -- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ac415b480 msgr2=0x7f2ac40b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ac415b480 0x7f2ac40b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac37fe640 1 -- 172.21.15.111:0/991902113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009b30 con 0x7f2ac415edd0 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.053+0000 7f2ac27fc640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ac415b480 0x7f2ac40b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.054+0000 7f2ac37fe640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 0x7f2ac40af270 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b450 tx=0x7f2abc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.054+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc011890 con 0x7f2ac415edd0 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.054+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc0119f0 con 0x7f2ac415edd0 2023-12-04T05:40:01.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.054+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc011d80 con 0x7f2ac415edd0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.054+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac40c8b90 con 0x7f2ac415edd0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.054+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac40c9150 con 0x7f2ac415edd0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.055+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac40c9670 con 0x7f2ac415edd0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.055+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f2abc01a400 con 0x7f2ac415edd0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.056+0000 7f2aab7fe640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ab006b2a0 0x7f2ab006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.056+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ab006b2a0 0x7f2ab006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.056+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f2abc089fb0 con 0x7f2ac415edd0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.056+0000 7f2ac2ffd640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ab006b2a0 0x7f2ab006d760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4035000 tx=0x7f2ab4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:01.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.060+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2abc0bce90 con 0x7f2ac415edd0 2023-12-04T05:40:01.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.260+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2ac4067460 con 0x7f2ac415edd0 2023-12-04T05:40:01.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.933+0000 7f2aab7fe640 1 -- 172.21.15.111:0/991902113 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2abc0150a0 con 0x7f2ac415edd0 2023-12-04T05:40:01.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ab006b2a0 msgr2=0x7f2ab006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ab006b2a0 0x7f2ab006d760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4035000 tx=0x7f2ab4030000 comp rx=0 tx=0).stop 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 msgr2=0x7f2ac40af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 0x7f2ac40af270 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b450 tx=0x7f2abc00b910 comp rx=0 tx=0).stop 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 shutdown_connections 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2ab006b2a0 0x7f2ab006d760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac415edd0 0x7f2ac40af270 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ac415b480 0x7f2ac40b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 --2- 172.21.15.111:0/991902113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ac40cf9f0 0x7f2ac40c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 >> 172.21.15.111:0/991902113 conn(0x7f2ac40b2a70 msgr2=0x7f2ac40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 shutdown_connections 2023-12-04T05:40:01.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:01.936+0000 7f2ac951d640 1 -- 172.21.15.111:0/991902113 wait complete. 2023-12-04T05:40:01.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-04T05:40:02.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.101+0000 7f8037a2d640 1 Processor -- start 2023-12-04T05:40:02.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.102+0000 7f8037a2d640 1 -- start start 2023-12-04T05:40:02.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.102+0000 7f8037a2d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f803015d480 0x7f80300d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.102+0000 7f8037a2d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300cfaf0 con 0x7f80300d10f0 2023-12-04T05:40:02.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.102+0000 7f8037a2d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300d1530 con 0x7f80300d1e80 2023-12-04T05:40:02.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.102+0000 7f8037a2d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300d3fa0 con 0x7f803015d480 2023-12-04T05:40:02.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8034fc8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f80300d10f0 0x7f80300d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47232/0 (socket says 172.21.15.111:47232) 2023-12-04T05:40:02.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8034fc8640 1 -- 172.21.15.111:0/2785441659 learned_addr learned my addr 172.21.15.111:0/2785441659 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:02.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539233670 0 0) 0x7f80300cfaf0 con 0x7f80300d10f0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f80357c9640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f803015d480 0x7f80300d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f801c003520 con 0x7f80300d10f0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497639473 0 0) 0x7f801c003520 con 0x7f80300d10f0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f803015d480 msgr2=0x7f80300d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f803015d480 0x7f80300d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 >> v1:172.21.15.192:6789/0 conn(0x7f80300d1e80 legacy=0x7f80300afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300cfdb0 con 0x7f80300d10f0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f8020001670 con 0x7f80300d10f0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 >> v1:172.21.15.111:6789/0 conn(0x7f80300d10f0 legacy=0x7f80300d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f801c0039a0 0x7f801c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0047c0 0x7f801c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c005180 0x7f801c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300d3fa0 con 0x7f801c0039a0 2023-12-04T05:40:02.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c00c020 con 0x7f801c0047c0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c00c1e0 con 0x7f801c005180 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f80357c9640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f803015d480 0x7f80300d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f80357c9640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f801c0039a0 0x7f801c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8034fc8640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0047c0 0x7f801c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.103+0000 7f8035fca640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c005180 0x7f801c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f80357c9640 1 -- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c005180 msgr2=0x7f801c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f80357c9640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c005180 0x7f801c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f80357c9640 1 -- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0047c0 msgr2=0x7f801c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f80357c9640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c0047c0 0x7f801c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f80357c9640 1 -- 172.21.15.111:0/2785441659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c003520 con 0x7f801c0039a0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f80357c9640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f801c0039a0 0x7f801c003dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f803008bc50 tx=0x7f801c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db36d2dfebc50619 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.104+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c02cc80 con 0x7f801c0039a0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c02cde0 con 0x7f801c0039a0 2023-12-04T05:40:02.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8027fff640 1 -- 172.21.15.111:0/2785441659 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c003bb0 con 0x7f801c0039a0 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8037a2d640 1 -- 172.21.15.111:0/2785441659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f801c0039a0 msgr2=0x7f801c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8037a2d640 1 --2- 172.21.15.111:0/2785441659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f801c0039a0 0x7f801c003dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f803008bc50 tx=0x7f801c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8035fca640 1 -- 172.21.15.111:0/2785441659 reap_dead start 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8037a2d640 1 -- 172.21.15.111:0/2785441659 shutdown_connections 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8037a2d640 1 -- 172.21.15.111:0/2785441659 >> 172.21.15.111:0/2785441659 conn(0x7f803009c4d0 msgr2=0x7f80300d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8037a2d640 1 -- 172.21.15.111:0/2785441659 shutdown_connections 2023-12-04T05:40:02.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.105+0000 7f8037a2d640 1 -- 172.21.15.111:0/2785441659 wait complete. 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.106+0000 7f8037a2d640 1 Processor -- start 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.106+0000 7f8037a2d640 1 -- start start 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8037a2d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c0039a0 0x7f8030152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8037a2d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80300ccf40 0x7f80300b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8037a2d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 0x7f80300ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8037a2d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b7e00 con 0x7f80300b7940 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8037a2d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b80e0 con 0x7f80300ccf40 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8037a2d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b83c0 con 0x7f801c0039a0 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 0x7f80300ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 0x7f80300ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40984/0 (socket says 172.21.15.111:40984) 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 -- 172.21.15.111:0/3095804011 learned_addr learned my addr 172.21.15.111:0/3095804011 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8034fc8640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80300ccf40 0x7f80300b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f80357c9640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c0039a0 0x7f8030152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 -- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c0039a0 msgr2=0x7f8030152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c0039a0 0x7f8030152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 -- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80300ccf40 msgr2=0x7f80300b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80300ccf40 0x7f80300b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.107+0000 7f8035fca640 1 -- 172.21.15.111:0/3095804011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300b86a0 con 0x7f80300b7940 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8034fc8640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80300ccf40 0x7f80300b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f80357c9640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c0039a0 0x7f8030152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8035fca640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 0x7f80300ca690 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f80300ca1f0 tx=0x7f802800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280053a0 con 0x7f80300b7940 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80280035f0 con 0x7f80300b7940 2023-12-04T05:40:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80300b8980 con 0x7f80300b7940 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280039a0 con 0x7f80300b7940 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.108+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80300cadb0 con 0x7f80300b7940 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.109+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80300cafd0 con 0x7f80300b7940 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.110+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f8028003b00 con 0x7f80300b7940 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.110+0000 7f8025ffb640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f801406b2a0 0x7f801406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.110+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f80280898d0 con 0x7f80300b7940 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.110+0000 7f80357c9640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f801406b2a0 0x7f801406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.110+0000 7f80357c9640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f801406b2a0 0x7f801406d760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f802c036000 tx=0x7f802c039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:02.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.114+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8028054670 con 0x7f80300b7940 2023-12-04T05:40:02.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:02.282+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8030069360 con 0x7f80300b7940 2023-12-04T05:40:03.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.197+0000 7f8025ffb640 1 -- 172.21.15.111:0/3095804011 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8028057c90 con 0x7f80300b7940 2023-12-04T05:40:03.199 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f801406b2a0 msgr2=0x7f801406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f801406b2a0 0x7f801406d760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f802c036000 tx=0x7f802c039040 comp rx=0 tx=0).stop 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 msgr2=0x7f80300ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 0x7f80300ca690 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f80300ca1f0 tx=0x7f802800f920 comp rx=0 tx=0).stop 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 shutdown_connections 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f801406b2a0 0x7f801406d760 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80300b7940 0x7f80300ca690 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80300ccf40 0x7f80300b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 --2- 172.21.15.111:0/3095804011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f801c0039a0 0x7f8030152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 >> 172.21.15.111:0/3095804011 conn(0x7f803009c4d0 msgr2=0x7f80300d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:03.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 shutdown_connections 2023-12-04T05:40:03.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.200+0000 7f8037a2d640 1 -- 172.21.15.111:0/3095804011 wait complete. 2023-12-04T05:40:03.214 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-04T05:40:03.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.319+0000 7f59e95a0640 1 Processor -- start 2023-12-04T05:40:03.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.319+0000 7f59e95a0640 1 -- start start 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e95a0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59e413fd70 0x7f59e415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e95a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e4140150 0x7f59e406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e95a0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e40b37a0 con 0x7f59e406cc80 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e95a0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e406d130 con 0x7f59e413fd70 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e95a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e415e540 con 0x7f59e4140150 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59e413fd70 0x7f59e415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59e413fd70 0x7f59e415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40988/0 (socket says 172.21.15.111:40988) 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.320+0000 7f59e3fff640 1 -- 172.21.15.111:0/3753376761 learned_addr learned my addr 172.21.15.111:0/3753376761 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:03.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e3fff640 1 -- 172.21.15.111:0/3753376761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e4140150 msgr2=0x7f59e406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:03.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e3fff640 1 --2- 172.21.15.111:0/3753376761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e4140150 0x7f59e406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e3fff640 1 -- 172.21.15.111:0/3753376761 >> v1:172.21.15.192:6789/0 conn(0x7f59e406cc80 legacy=0x7f59e40b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:03.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e3fff640 1 -- 172.21.15.111:0/3753376761 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4142ba0 con 0x7f59e413fd70 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e3fff640 1 --2- 172.21.15.111:0/3753376761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e413fd70 0x7f59e415e000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009870 tx=0x7f59d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2146a312c692e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e2ffd640 1 -- 172.21.15.111:0/3753376761 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4031640 con 0x7f59e413fd70 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e2ffd640 1 -- 172.21.15.111:0/3753376761 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40317a0 con 0x7f59e413fd70 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.321+0000 7f59e2ffd640 1 -- 172.21.15.111:0/3753376761 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4031b30 con 0x7f59e413fd70 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 -- 172.21.15.111:0/3753376761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e413fd70 msgr2=0x7f59e415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 --2- 172.21.15.111:0/3753376761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e413fd70 0x7f59e415e000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009870 tx=0x7f59d402b8f0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 -- 172.21.15.111:0/3753376761 shutdown_connections 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 --2- 172.21.15.111:0/3753376761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e4140150 0x7f59e406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 --2- 172.21.15.111:0/3753376761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e413fd70 0x7f59e415e000 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 -- 172.21.15.111:0/3753376761 >> 172.21.15.111:0/3753376761 conn(0x7f59e406a6a0 msgr2=0x7f59e406aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 -- 172.21.15.111:0/3753376761 shutdown_connections 2023-12-04T05:40:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.322+0000 7f59e95a0640 1 -- 172.21.15.111:0/3753376761 wait complete. 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.323+0000 7f59e95a0640 1 Processor -- start 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.323+0000 7f59e95a0640 1 -- start start 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e95a0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 0x7f59e41e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e95a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e413fd70 0x7f59e41e9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e95a0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59e4140150 0x7f59e4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e95a0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41e98d0 con 0x7f59e406cc80 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e95a0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4156180 con 0x7f59e4140150 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e95a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e41e82f0 con 0x7f59e413fd70 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 0x7f59e41e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:03.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 0x7f59e41e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41000/0 (socket says 172.21.15.111:41000) 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 -- 172.21.15.111:0/1701524176 learned_addr learned my addr 172.21.15.111:0/1701524176 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e37fe640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e413fd70 0x7f59e41e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e8d9f640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59e4140150 0x7f59e4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 -- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e413fd70 msgr2=0x7f59e41e9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e413fd70 0x7f59e41e9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 -- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59e4140150 msgr2=0x7f59e4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59e4140150 0x7f59e4155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e3fff640 1 -- 172.21.15.111:0/1701524176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40094d0 con 0x7f59e406cc80 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.324+0000 7f59e8d9f640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59e4140150 0x7f59e4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.325+0000 7f59e3fff640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 0x7f59e41e7c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009870 tx=0x7f59d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.325+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d403ac60 con 0x7f59e406cc80 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.325+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d403adc0 con 0x7f59e406cc80 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.325+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e41e85d0 con 0x7f59e406cc80 2023-12-04T05:40:03.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.325+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4044630 con 0x7f59e406cc80 2023-12-04T05:40:03.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.325+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e41e8b90 con 0x7f59e406cc80 2023-12-04T05:40:03.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.326+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e40729b0 con 0x7f59e406cc80 2023-12-04T05:40:03.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.326+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f59d402f050 con 0x7f59e406cc80 2023-12-04T05:40:03.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.326+0000 7f59e0ff9640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f59d006b2a0 0x7f59d006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:03.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.326+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f59d40aba70 con 0x7f59e406cc80 2023-12-04T05:40:03.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.326+0000 7f59e37fe640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f59d006b2a0 0x7f59d006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:03.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.327+0000 7f59e37fe640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f59d006b2a0 0x7f59d006d760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59e415cd80 tx=0x7f59d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:03.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.331+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59d4076a50 con 0x7f59e406cc80 2023-12-04T05:40:03.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:03.498+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59e404e960 con 0x7f59e406cc80 2023-12-04T05:40:04.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.358+0000 7f59e0ff9640 1 -- 172.21.15.111:0/1701524176 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f59d4031e60 con 0x7f59e406cc80 2023-12-04T05:40:04.424 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2023-12-04T05:40:04.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f59d006b2a0 msgr2=0x7f59d006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f59d006b2a0 0x7f59d006d760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59e415cd80 tx=0x7f59d8008040 comp rx=0 tx=0).stop 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 msgr2=0x7f59e41e7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 0x7f59e41e7c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009870 tx=0x7f59d40046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 shutdown_connections 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f59d006b2a0 0x7f59d006d760 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59e4140150 0x7f59e4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59e413fd70 0x7f59e41e9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 --2- 172.21.15.111:0/1701524176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e406cc80 0x7f59e41e7c60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 >> 172.21.15.111:0/1701524176 conn(0x7f59e406a6a0 msgr2=0x7f59e4072250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 shutdown_connections 2023-12-04T05:40:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.360+0000 7f59e95a0640 1 -- 172.21.15.111:0/1701524176 wait complete. 2023-12-04T05:40:04.427 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-04T05:40:04.427 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-04T05:40:04.427 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-04T05:40:04.427 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:04.427 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-04T05:40:04.427 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-04T05:40:04.428 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-04T05:40:04.428 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:04.428 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-04T05:40:04.428 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-04T05:40:04.428 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-04T05:40:04.428 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:04.428 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-04T05:40:04.428 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-04T05:40:04.429 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-04T05:40:04.429 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:04.438 INFO:tasks.ceph.mds.c:Stopped 2023-12-04T05:40:04.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-04T05:40:04.439 INFO:tasks.ceph.mds.a:Stopped 2023-12-04T05:40:04.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-04T05:40:04.440 INFO:tasks.ceph.mds.b:Stopped 2023-12-04T05:40:04.441 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-04T05:40:04.441 INFO:tasks.ceph.mds.d:Stopped 2023-12-04T05:40:04.442 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-04T05:40:04.481 INFO:tasks.ceph.mds.c:Started 2023-12-04T05:40:04.482 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:04.483 INFO:tasks.ceph.mds.a:Started 2023-12-04T05:40:04.483 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:04.484 INFO:tasks.ceph.mds.b:Started 2023-12-04T05:40:04.484 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:04.485 INFO:tasks.ceph.mds.d:Started 2023-12-04T05:40:04.485 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:04.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-04T05:40:04.560 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2023-12-04T05:40:04.560 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2023-12-04T05:40:04.569 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-04T05:40:04.569 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-04T05:40:04.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.589+0000 7f5aaa05c640 1 Processor -- start 2023-12-04T05:40:04.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.589+0000 7f5aaa05c640 1 -- start start 2023-12-04T05:40:04.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aaa05c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 0x7f5aa40ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aaa05c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa40cdcb0 con 0x7f5aa40d1780 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aaa05c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa40d39e0 con 0x7f5aa40b20a0 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aaa05c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40d0610 con 0x7f5aa40d01e0 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5aa40d1780 0x7f5aa40d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47248/0 (socket says 172.21.15.111:47248) 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa37fe640 1 -- 172.21.15.111:0/1754862372 learned_addr learned my addr 172.21.15.111:0/1754862372 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa3fff640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 0x7f5aa40ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889373984 0 0) 0x7f5aa40cdcb0 con 0x7f5aa40d1780 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a8c003520 con 0x7f5aa40d1780 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226404532 0 0) 0x7f5a8c003520 con 0x7f5aa40d1780 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 msgr2=0x7f5aa40ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 0x7f5aa40ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 >> v1:172.21.15.192:6789/0 conn(0x7f5aa40b20a0 legacy=0x7f5aa40b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40d3cc0 con 0x7f5aa40d1780 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f5a98001670 con 0x7f5aa40d1780 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 >> v1:172.21.15.111:6789/0 conn(0x7f5aa40d1780 legacy=0x7f5aa40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 0x7f5a8c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5a8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a8c004d00 0x7f5a8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40d0610 con 0x7f5a8c003f20 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c00bb10 con 0x7f5a8c004d00 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c00bcd0 con 0x7f5aa40d01e0 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5a8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa37fe640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 0x7f5a8c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.590+0000 7f5aa3fff640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a8c004d00 0x7f5a8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa2ffd640 1 -- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 msgr2=0x7f5a8c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa40d01e0 0x7f5a8c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa2ffd640 1 -- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a8c004d00 msgr2=0x7f5a8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a8c004d00 0x7f5a8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa2ffd640 1 -- 172.21.15.111:0/1754862372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c003520 con 0x7f5a8c003f20 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5a8c0047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001750 tx=0x7f5a8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b6670e25774641 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940052d0 con 0x7f5a8c003f20 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9402e050 con 0x7f5a8c003f20 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.591+0000 7f5aa27fc640 1 -- 172.21.15.111:0/1754862372 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940035f0 con 0x7f5a8c003f20 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1754862372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 msgr2=0x7f5a8c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1754862372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5a8c0047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001750 tx=0x7f5a8c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aa3fff640 1 -- 172.21.15.111:0/1754862372 reap_dead start 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1754862372 shutdown_connections 2023-12-04T05:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1754862372 >> 172.21.15.111:0/1754862372 conn(0x7f5aa409c4e0 msgr2=0x7f5aa40d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1754862372 shutdown_connections 2023-12-04T05:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.592+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1754862372 wait complete. 2023-12-04T05:40:04.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.593+0000 7f5aaa05c640 1 Processor -- start 2023-12-04T05:40:04.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.593+0000 7f5aaa05c640 1 -- start start 2023-12-04T05:40:04.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aaa05c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5aa40c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aaa05c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40b1210 0x7f5aa41f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aaa05c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa41f2e20 0x7f5aa40b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aaa05c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40b6b70 con 0x7f5a8c003f20 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aaa05c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40c7190 con 0x7f5aa40b1210 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aaa05c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40c96c0 con 0x7f5aa41f2e20 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aa37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5aa40c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aa37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5aa40c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41048/0 (socket says 172.21.15.111:41048) 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aa37fe640 1 -- 172.21.15.111:0/1580943728 learned_addr learned my addr 172.21.15.111:0/1580943728 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40b1210 0x7f5aa41f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.594+0000 7f5aa3fff640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa41f2e20 0x7f5aa40b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aa37fe640 1 -- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa41f2e20 msgr2=0x7f5aa40b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aa37fe640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa41f2e20 0x7f5aa40b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aa37fe640 1 -- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40b1210 msgr2=0x7f5aa41f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aa37fe640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40b1210 0x7f5aa41f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:04.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aa37fe640 1 -- 172.21.15.111:0/1580943728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40c99a0 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aa37fe640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5aa40c6c50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5a980072c0 tx=0x7f5a98007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800f8e0 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800fa40 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa40b6780 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800fdf0 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.595+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa40b74d0 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.596+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa40b6de0 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.596+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5a9801d460 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.597+0000 7f5a8bfff640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5a9006b2d0 0x7f5a9006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.597+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a98089800 con 0x7f5a8c003f20 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.597+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5a9006b2d0 0x7f5a9006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.597+0000 7f5aa2ffd640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5a9006b2d0 0x7f5a9006d790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001750 tx=0x7f5a94009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.601+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a98054ab0 con 0x7f5a8c003f20 2023-12-04T05:40:04.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:04.779+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5aa40c9b30 con 0x7f5a8c003f20 2023-12-04T05:40:05.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.367+0000 7f5a8bfff640 1 -- 172.21.15.111:0/1580943728 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5a980580d0 con 0x7f5a8c003f20 2023-12-04T05:40:05.369 INFO:teuthology.orchestra.run.smithi111.stderr: removed all blocklist entries 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5a9006b2d0 msgr2=0x7f5a9006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5a9006b2d0 0x7f5a9006d790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001750 tx=0x7f5a94009030 comp rx=0 tx=0).stop 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 msgr2=0x7f5aa40c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5aa40c6c50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5a980072c0 tx=0x7f5a98007780 comp rx=0 tx=0).stop 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 shutdown_connections 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5a9006b2d0 0x7f5a9006d790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa41f2e20 0x7f5aa40b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa40b1210 0x7f5aa41f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 --2- 172.21.15.111:0/1580943728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a8c003f20 0x7f5aa40c6c50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 >> 172.21.15.111:0/1580943728 conn(0x7f5aa409c4e0 msgr2=0x7f5aa40b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 shutdown_connections 2023-12-04T05:40:05.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.370+0000 7f5aaa05c640 1 -- 172.21.15.111:0/1580943728 wait complete. 2023-12-04T05:40:05.385 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-04T05:40:05.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.485+0000 7f3f64215640 1 Processor -- start 2023-12-04T05:40:05.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.485+0000 7f3f64215640 1 -- start start 2023-12-04T05:40:05.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.486+0000 7f3f64215640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3f5c15b480 0x7f3f5c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.486+0000 7f3f64215640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0cbca0 con 0x7f3f5c15f4e0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.486+0000 7f3f64215640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0cbf40 con 0x7f3f5c0cf9f0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.486+0000 7f3f64215640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cc220 con 0x7f3f5c15b480 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.486+0000 7f3f627b2640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3f5c15f4e0 0x7f3f5c0b08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47256/0 (socket says 172.21.15.111:47256) 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.486+0000 7f3f627b2640 1 -- 172.21.15.111:0/111556503 learned_addr learned my addr 172.21.15.111:0/111556503 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f61fb1640 1 --2- 172.21.15.111:0/111556503 >> v2:172.21.15.192:3300/0 conn(0x7f3f5c15b480 0x7f3f5c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296958707 0 0) 0x7f3f5c0cbca0 con 0x7f3f5c15f4e0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f48003520 con 0x7f3f5c15f4e0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570133449 0 0) 0x7f3f48003520 con 0x7f3f5c15f4e0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 >> v2:172.21.15.192:3300/0 conn(0x7f3f5c15b480 msgr2=0x7f3f5c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 --2- 172.21.15.111:0/111556503 >> v2:172.21.15.192:3300/0 conn(0x7f3f5c15b480 0x7f3f5c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 >> v1:172.21.15.192:6789/0 conn(0x7f3f5c0cf9f0 legacy=0x7f3f5c15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0cc500 con 0x7f3f5c15f4e0 2023-12-04T05:40:05.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f3f54001670 con 0x7f3f5c15f4e0 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 >> v1:172.21.15.111:6789/0 conn(0x7f3f5c15f4e0 legacy=0x7f3f5c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48003970 0x7f3f48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f480047c0 0x7f3f48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f48005180 0x7f3f4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cc220 con 0x7f3f48005180 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4800c020 con 0x7f3f48003970 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4800c1e0 con 0x7f3f480047c0 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f627b2640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f48005180 0x7f3f4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f61fb1640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48003970 0x7f3f48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f617b0640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f480047c0 0x7f3f48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f627b2640 1 -- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f480047c0 msgr2=0x7f3f48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f627b2640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f480047c0 0x7f3f48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f627b2640 1 -- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48003970 msgr2=0x7f3f48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:05.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f627b2640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48003970 0x7f3f48003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.487+0000 7f3f627b2640 1 -- 172.21.15.111:0/111556503 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48003520 con 0x7f3f48005180 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.488+0000 7f3f627b2640 1 -- 172.21.15.111:0/111556503 reap_dead start 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.488+0000 7f3f627b2640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f48005180 0x7f3f4800b980 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400c240 tx=0x7f3f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da564bae45af9c64 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.488+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54005ad0 con 0x7f3f48005180 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.488+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54005c70 con 0x7f3f48005180 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.488+0000 7f3f60faf640 1 -- 172.21.15.111:0/111556503 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5402c8b0 con 0x7f3f48005180 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 -- 172.21.15.111:0/111556503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f48005180 msgr2=0x7f3f4800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f48005180 0x7f3f4800b980 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400c240 tx=0x7f3f480057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 -- 172.21.15.111:0/111556503 shutdown_connections 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 --2- 172.21.15.111:0/111556503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f48005180 0x7f3f4800b980 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 -- 172.21.15.111:0/111556503 >> 172.21.15.111:0/111556503 conn(0x7f3f5c0b2a70 msgr2=0x7f3f5c0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 -- 172.21.15.111:0/111556503 shutdown_connections 2023-12-04T05:40:05.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.489+0000 7f3f64215640 1 -- 172.21.15.111:0/111556503 wait complete. 2023-12-04T05:40:05.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.490+0000 7f3f64215640 1 Processor -- start 2023-12-04T05:40:05.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.490+0000 7f3f64215640 1 -- start start 2023-12-04T05:40:05.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f64215640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48005180 0x7f3f5c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f64215640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 0x7f3f5c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f64215640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0ca330 0x7f3f5c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f64215640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c5aa0 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f64215640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c155fd0 con 0x7f3f48005180 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f64215640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c8f30 con 0x7f3f5c0ca330 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 0x7f3f5c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 0x7f3f5c0c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41056/0 (socket says 172.21.15.111:41056) 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 -- 172.21.15.111:0/1319069454 learned_addr learned my addr 172.21.15.111:0/1319069454 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f627b2640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0ca330 0x7f3f5c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f61fb1640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48005180 0x7f3f5c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 -- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0ca330 msgr2=0x7f3f5c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0ca330 0x7f3f5c0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 -- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48005180 msgr2=0x7f3f5c155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48005180 0x7f3f5c155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 -- 172.21.15.111:0/1319069454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0c67c0 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f617b0640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 0x7f3f5c0c89f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00e480 tx=0x7f3f4c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.491+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0052e0 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.492+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c005440 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.492+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0c5e90 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.492+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c003830 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.492+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c0c6450 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.492+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0c6950 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.493+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3f4c003990 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.493+0000 7f3f527fc640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f4006b2d0 0x7f3f4006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.493+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f4c0893d0 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.493+0000 7f3f61fb1640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f4006b2d0 0x7f3f4006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.494+0000 7f3f61fb1640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f4006b2d0 0x7f3f4006d790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f58014450 tx=0x7f3f5800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:05.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.497+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f4c0546d0 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.728+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f5c0c4690 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.728+0000 7f3f527fc640 1 -- 172.21.15.111:0/1319069454 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3f4c05e070 con 0x7f3f5c0c55a0 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "auth_dump": [ 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.a", 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lpb6cJBAA7iF+mExHXiopk7JbrjCW1g==", 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:05.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.b", 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1l+X3+MBAAQatwhQBZ5/VZ0xuNJpQSYA==", 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.731 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.c", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lgo37KxAA/89DL0NuA+KyX+CUbzzU+A==", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.d", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lPSdXOBAAGSdsougjKMP1afwkK/2gmw==", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.732 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.0", 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDOZW1lxDi6HRAA0pl+LbQDgGxT9fr9vGzWHA==", 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.733 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.1", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDRZW1lu/JsHRAApgnvWoeQ9MAOPdBuFifdvA==", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.2", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDUZW1ldy+tGxAAhPf6cR9pyGpyD0cAy1nd5g==", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.734 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.3", 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDXZW1lDCs7GRAAADDCyZOB4a+fmmWEgezmxw==", 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.4", 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDdZW1lzq4XJRAAG90v2DSwzGMnGUEK14Z7ew==", 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.5", 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDgZW1lvzQUJBAAYrLHQV/+N0irpVvSJh1LkQ==", 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.736 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.6", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDjZW1ll6sCIhAAlUiDCgD51TZDbjgznyHYSw==", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.7", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDmZW1ll2Z/HxAALDVVTrBx3aaiRVf/Is2p5Q==", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:05.737 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.0", 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1l+5T3ABAA7h+0Ksz9xnrfFZKgsBbhIA==", 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.738 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.1", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1lqipMChAAXB0Z40yv2jFsOlHuH8i4Pg==", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.739 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.2", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1lwejPBhAAVmTVqunSjtPrXqLWy4n1AA==", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.3", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1lrQobEBAAAJ8wvchn2R7AvCfLR5Me0w==", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:05.740 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.admin", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1l+r3tBxAA2OQ02bhGgalFabd6uZY6Ww==", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.741 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mds", 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDsZW1lRa07GRAAoCqMfJk0UB9WJQJitseC7w==", 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mds" 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mgr", 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDsZW1lBrY7GRAAVkSnCUEqxNzKgx6s1gFs8g==", 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.742 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-osd", 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDsZW1lDb07GRAAVXKqotyIn+bGLpER/8QB6w==", 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-osd" 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd", 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDsZW1lzsM7GRAA9CEIv94f6kM4iqiQvm+d+Q==", 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.743 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDsZW1lzso7GRAAw15zpSeu1YvUQ4OyUrV44g==", 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rgw", 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDsZW1lrRI8GRAAEe6QnxRJyyivhpmILVAhRw==", 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.x", 2023-12-04T05:40:05.744 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1l8LA1IRAAZmq/pSux9bgsmwY5LTOzIQ==", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.y", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lztwVHRAAHxYfbr/N8aCdSCIPzjqzXw==", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.745 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:05.746 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2023-12-04T05:40:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:40:05.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f4006b2d0 msgr2=0x7f3f4006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f4006b2d0 0x7f3f4006d790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f58014450 tx=0x7f3f5800b040 comp rx=0 tx=0).stop 2023-12-04T05:40:05.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 msgr2=0x7f3f5c0c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 0x7f3f5c0c89f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00e480 tx=0x7f3f4c00e940 comp rx=0 tx=0).stop 2023-12-04T05:40:05.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 shutdown_connections 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3f4006b2d0 0x7f3f4006d790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0ca330 0x7f3f5c0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0c55a0 0x7f3f5c0c89f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 --2- 172.21.15.111:0/1319069454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f48005180 0x7f3f5c155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 >> 172.21.15.111:0/1319069454 conn(0x7f3f5c0b2a70 msgr2=0x7f3f5c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 shutdown_connections 2023-12-04T05:40:05.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.730+0000 7f3f64215640 1 -- 172.21.15.111:0/1319069454 wait complete. 2023-12-04T05:40:05.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-04T05:40:05.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.847+0000 7f34ac075640 1 Processor -- start 2023-12-04T05:40:05.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.847+0000 7f34ac075640 1 -- start start 2023-12-04T05:40:05.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34ac075640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f34a413cee0 0x7f34a406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34ac075640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f34a4140200 0x7f34a406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34ac075640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a40aea20 con 0x7f34a415a490 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34ac075640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a413d2c0 con 0x7f34a4140200 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34ac075640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a415b3a0 con 0x7f34a413cee0 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f34a4140200 0x7f34a406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f34a4140200 0x7f34a406c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41066/0 (socket says 172.21.15.111:41066) 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34aa612640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f34a415a490 0x7f34a415a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47260/0 (socket says 172.21.15.111:47260) 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 -- 172.21.15.111:0/1358395583 learned_addr learned my addr 172.21.15.111:0/1358395583 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9e11640 1 --2- 172.21.15.111:0/1358395583 >> v2:172.21.15.192:3300/0 conn(0x7f34a413cee0 0x7f34a406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 -- 172.21.15.111:0/1358395583 >> v2:172.21.15.192:3300/0 conn(0x7f34a413cee0 msgr2=0x7f34a406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:05.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 --2- 172.21.15.111:0/1358395583 >> v2:172.21.15.192:3300/0 conn(0x7f34a413cee0 0x7f34a406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 -- 172.21.15.111:0/1358395583 >> v1:172.21.15.111:6789/0 conn(0x7f34a415a490 legacy=0x7f34a415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a8e0f640 1 -- 172.21.15.111:0/1358395583 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138733036 0 0) 0x7f34a40aea20 con 0x7f34a415a490 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.848+0000 7f34a9610640 1 -- 172.21.15.111:0/1358395583 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a415b0c0 con 0x7f34a4140200 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.849+0000 7f34a9610640 1 --2- 172.21.15.111:0/1358395583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a406c790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f34a40b2280 tx=0x7f349402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141aa35171464055 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.849+0000 7f34a8e0f640 1 -- 172.21.15.111:0/1358395583 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349402ee90 con 0x7f34a4140200 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.849+0000 7f34a8e0f640 1 -- 172.21.15.111:0/1358395583 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3494036440 con 0x7f34a4140200 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.849+0000 7f34a8e0f640 1 -- 172.21.15.111:0/1358395583 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34940367d0 con 0x7f34a4140200 2023-12-04T05:40:05.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 -- 172.21.15.111:0/1358395583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 msgr2=0x7f34a406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 --2- 172.21.15.111:0/1358395583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a406c790 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f34a40b2280 tx=0x7f349402e0e0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 -- 172.21.15.111:0/1358395583 shutdown_connections 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 --2- 172.21.15.111:0/1358395583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a406c790 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 --2- 172.21.15.111:0/1358395583 >> v2:172.21.15.192:3300/0 conn(0x7f34a413cee0 0x7f34a406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 -- 172.21.15.111:0/1358395583 >> 172.21.15.111:0/1358395583 conn(0x7f34a406a4c0 msgr2=0x7f34a406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 -- 172.21.15.111:0/1358395583 shutdown_connections 2023-12-04T05:40:05.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.850+0000 7f34ac075640 1 -- 172.21.15.111:0/1358395583 wait complete. 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.851+0000 7f34ac075640 1 Processor -- start 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.851+0000 7f34ac075640 1 -- start start 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a413cee0 0x7f34a413f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a4136160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a415a490 0x7f34a41359b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40b0a70 con 0x7f34a4140200 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40b04b0 con 0x7f34a413cee0 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40b0790 con 0x7f34a415a490 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a4136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a4136160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41070/0 (socket says 172.21.15.111:41070) 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 -- 172.21.15.111:0/1589418473 learned_addr learned my addr 172.21.15.111:0/1589418473 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:05.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 -- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a415a490 msgr2=0x7f34a41359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a415a490 0x7f34a41359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 -- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a413cee0 msgr2=0x7f34a413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9e11640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a413cee0 0x7f34a413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a413cee0 0x7f34a413f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 -- 172.21.15.111:0/1589418473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349400b770 con 0x7f34a4140200 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34a9610640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a4136160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3494000c00 tx=0x7f34940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494004230 con 0x7f34a4140200 2023-12-04T05:40:05.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3494004390 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a414d7f0 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494042960 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.852+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a4135ef0 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.853+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a413c720 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.854+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3494042ac0 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.854+0000 7f349a7fc640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f348806b2a0 0x7f348806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.854+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f34940aed10 con 0x7f34a4140200 2023-12-04T05:40:05.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.854+0000 7f34a9e11640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f348806b2a0 0x7f348806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:05.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.855+0000 7f34a9e11640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f348806b2a0 0x7f348806d760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34a415a030 tx=0x7f34a000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:05.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:05.858+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3494079f60 con 0x7f34a4140200 2023-12-04T05:40:06.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.089+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f34a40703d0 con 0x7f34a4140200 2023-12-04T05:40:06.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.091+0000 7f349a7fc640 1 -- 172.21.15.111:0/1589418473 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f34a40703d0 con 0x7f34a4140200 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f348806b2a0 msgr2=0x7f348806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f348806b2a0 0x7f348806d760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34a415a030 tx=0x7f34a000b040 comp rx=0 tx=0).stop 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 msgr2=0x7f34a4136160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a4136160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3494000c00 tx=0x7f34940046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 shutdown_connections 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f348806b2a0 0x7f348806d760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a415a490 0x7f34a41359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34a4140200 0x7f34a4136160 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 --2- 172.21.15.111:0/1589418473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a413cee0 0x7f34a413f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 >> 172.21.15.111:0/1589418473 conn(0x7f34a406a4c0 msgr2=0x7f34a415d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.093+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 shutdown_connections 2023-12-04T05:40:06.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.094+0000 7f34ac075640 1 -- 172.21.15.111:0/1589418473 wait complete. 2023-12-04T05:40:06.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-04T05:40:06.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.204+0000 7fbd20a90640 1 Processor -- start 2023-12-04T05:40:06.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.204+0000 7fbd20a90640 1 -- start start 2023-12-04T05:40:06.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd20a90640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd1c06cae0 0x7fbd1c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd20a90640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15dab0 0x7fbd1c0703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd20a90640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1c154220 con 0x7fbd1c0b3310 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd20a90640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd1c15a3c0 con 0x7fbd1c06cae0 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd20a90640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd1c15d2f0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd1c06cae0 0x7fbd1c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd13fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbd1c0b3310 0x7fbd1c159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47262/0 (socket says 172.21.15.111:47262) 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd1c06cae0 0x7fbd1c06cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41086/0 (socket says 172.21.15.111:41086) 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd13fff640 1 -- 172.21.15.111:0/1462149845 learned_addr learned my addr 172.21.15.111:0/1462149845 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1ad9d640 1 --2- 172.21.15.111:0/1462149845 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15dab0 0x7fbd1c0703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1a59c640 1 -- 172.21.15.111:0/1462149845 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15dab0 msgr2=0x7fbd1c0703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1a59c640 1 --2- 172.21.15.111:0/1462149845 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15dab0 0x7fbd1c0703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1a59c640 1 -- 172.21.15.111:0/1462149845 >> v1:172.21.15.111:6789/0 conn(0x7fbd1c0b3310 legacy=0x7fbd1c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1a59c640 1 -- 172.21.15.111:0/1462149845 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c06d400 con 0x7fbd1c06cae0 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd19d9b640 1 -- 172.21.15.111:0/1462149845 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033477815 0 0) 0x7fbd04001110 con 0x7fbd1c0b3310 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.205+0000 7fbd1ad9d640 1 --2- 172.21.15.111:0/1462149845 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15dab0 0x7fbd1c0703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:06.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.206+0000 7fbd1a59c640 1 --2- 172.21.15.111:0/1462149845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c06cae0 0x7fbd1c06cec0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c06e8e0 tx=0x7fbd0c02cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90efe4c8276cda06 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.206+0000 7fbd19d9b640 1 -- 172.21.15.111:0/1462149845 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c02db50 con 0x7fbd1c06cae0 2023-12-04T05:40:06.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.206+0000 7fbd19d9b640 1 -- 172.21.15.111:0/1462149845 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c02dcb0 con 0x7fbd1c06cae0 2023-12-04T05:40:06.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.206+0000 7fbd19d9b640 1 -- 172.21.15.111:0/1462149845 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c035660 con 0x7fbd1c06cae0 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 -- 172.21.15.111:0/1462149845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c06cae0 msgr2=0x7fbd1c06cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 --2- 172.21.15.111:0/1462149845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c06cae0 0x7fbd1c06cec0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c06e8e0 tx=0x7fbd0c02cda0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 -- 172.21.15.111:0/1462149845 shutdown_connections 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 --2- 172.21.15.111:0/1462149845 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15dab0 0x7fbd1c0703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 --2- 172.21.15.111:0/1462149845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c06cae0 0x7fbd1c06cec0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 -- 172.21.15.111:0/1462149845 >> 172.21.15.111:0/1462149845 conn(0x7fbd1c06a4c0 msgr2=0x7fbd1c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:06.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 -- 172.21.15.111:0/1462149845 shutdown_connections 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.207+0000 7fbd20a90640 1 -- 172.21.15.111:0/1462149845 wait complete. 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.208+0000 7fbd20a90640 1 Processor -- start 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.208+0000 7fbd20a90640 1 -- start start 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd20a90640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c06cae0 0x7fbd1c0ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd20a90640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd1c0b3310 0x7fbd1c148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd20a90640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 0x7fbd1c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd20a90640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c0abab0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd20a90640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c0a85e0 con 0x7fbd1c06cae0 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd20a90640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c0a88c0 con 0x7fbd1c0b3310 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 0x7fbd1c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 0x7fbd1c070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41092/0 (socket says 172.21.15.111:41092) 2023-12-04T05:40:06.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 -- 172.21.15.111:0/591865935 learned_addr learned my addr 172.21.15.111:0/591865935 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:06.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 -- 172.21.15.111:0/591865935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd1c0b3310 msgr2=0x7fbd1c148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd1c0b3310 0x7fbd1c148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 -- 172.21.15.111:0/591865935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c06cae0 msgr2=0x7fbd1c0ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c06cae0 0x7fbd1c0ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 -- 172.21.15.111:0/591865935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c00a5a0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd1ad9d640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 0x7fbd1c070480 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400cd50 tx=0x7fbd1400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.209+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400f680 con 0x7fbd1c15dab0 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.210+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1400f800 con 0x7fbd1c15dab0 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.210+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400fbb0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.210+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1c0a8ba0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.210+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c148cd0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.215 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.210+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1c0a90c0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.211+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbd1400b040 con 0x7fbd1c15dab0 2023-12-04T05:40:06.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.211+0000 7fbd12ffd640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbcfc06b2a0 0x7fbcfc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.211+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd1408a290 con 0x7fbd1c15dab0 2023-12-04T05:40:06.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.211+0000 7fbd1a59c640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbcfc06b2a0 0x7fbcfc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.212+0000 7fbd1a59c640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbcfc06b2a0 0x7fbcfc06d760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c037000 tx=0x7fbd0c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.216+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd14055450 con 0x7fbd1c15dab0 2023-12-04T05:40:06.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.449+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fbd1c0693d0 con 0x7fbd1c15dab0 2023-12-04T05:40:06.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.451+0000 7fbd12ffd640 1 -- 172.21.15.111:0/591865935 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd14058a70 con 0x7fbd1c15dab0 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbcfc06b2a0 msgr2=0x7fbcfc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbcfc06b2a0 0x7fbcfc06d760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c037000 tx=0x7fbd0c02f000 comp rx=0 tx=0).stop 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 msgr2=0x7fbd1c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 0x7fbd1c070480 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400cd50 tx=0x7fbd1400a650 comp rx=0 tx=0).stop 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 shutdown_connections 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbcfc06b2a0 0x7fbcfc06d760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1c15dab0 0x7fbd1c070480 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd1c0b3310 0x7fbd1c148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 --2- 172.21.15.111:0/591865935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c06cae0 0x7fbd1c0ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 >> 172.21.15.111:0/591865935 conn(0x7fbd1c06a4c0 msgr2=0x7fbd1c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:06.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 shutdown_connections 2023-12-04T05:40:06.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.454+0000 7fbd20a90640 1 -- 172.21.15.111:0/591865935 wait complete. 2023-12-04T05:40:06.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-04T05:40:06.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.565+0000 7efef411d640 1 Processor -- start 2023-12-04T05:40:06.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.565+0000 7efef411d640 1 -- start start 2023-12-04T05:40:06.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef411d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efeec06c960 0x7efeec15a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef411d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec0aea20 con 0x7efeec140200 2023-12-04T05:40:06.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef411d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec13d2c0 con 0x7efeec13cee0 2023-12-04T05:40:06.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef411d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec15b3a0 con 0x7efeec06c960 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef26ba640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7efeec140200 0x7efeec0cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47266/0 (socket says 172.21.15.111:47266) 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef26ba640 1 -- 172.21.15.111:0/49012126 learned_addr learned my addr 172.21.15.111:0/49012126 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.566+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598616976 0 0) 0x7efeec0aea20 con 0x7efeec140200 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed8003520 con 0x7efeec140200 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef16b8640 1 --2- 172.21.15.111:0/49012126 >> v2:172.21.15.192:3300/0 conn(0x7efeec06c960 0x7efeec15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 427947288 0 0) 0x7efed8003520 con 0x7efeec140200 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 >> v2:172.21.15.192:3300/0 conn(0x7efeec06c960 msgr2=0x7efeec15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 --2- 172.21.15.111:0/49012126 >> v2:172.21.15.192:3300/0 conn(0x7efeec06c960 0x7efeec15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 >> v1:172.21.15.192:6789/0 conn(0x7efeec13cee0 legacy=0x7efeec06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec15b0c0 con 0x7efeec140200 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7efee4001670 con 0x7efeec140200 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 >> v1:172.21.15.111:6789/0 conn(0x7efeec140200 legacy=0x7efeec0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed8003970 0x7efed8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efed80047c0 0x7efed8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8005180 0x7efed800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec15b3a0 con 0x7efed8003970 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed800c120 con 0x7efed8005180 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed800c2e0 con 0x7efed80047c0 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef1eb9640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed8003970 0x7efed8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef26ba640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8005180 0x7efed800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef16b8640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efed80047c0 0x7efed8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef1eb9640 1 -- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efed80047c0 msgr2=0x7efed8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef1eb9640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efed80047c0 0x7efed8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef1eb9640 1 -- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8005180 msgr2=0x7efed800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef1eb9640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8005180 0x7efed800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef26ba640 1 -- 172.21.15.111:0/49012126 reap_dead start 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.567+0000 7efef1eb9640 1 -- 172.21.15.111:0/49012126 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8003520 con 0x7efed8003970 2023-12-04T05:40:06.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.568+0000 7efef1eb9640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed8003970 0x7efed8003d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efee8001cb0 tx=0x7efed80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6470adb6b1a2db3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.568+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8003b70 con 0x7efed8003970 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.568+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8003cd0 con 0x7efed8003970 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.568+0000 7efef0eb7640 1 -- 172.21.15.111:0/49012126 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8005310 con 0x7efed8003970 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 -- 172.21.15.111:0/49012126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed8003970 msgr2=0x7efed8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed8003970 0x7efed8003d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efee8001cb0 tx=0x7efed80057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 -- 172.21.15.111:0/49012126 shutdown_connections 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 --2- 172.21.15.111:0/49012126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed8003970 0x7efed8003d90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 -- 172.21.15.111:0/49012126 >> 172.21.15.111:0/49012126 conn(0x7efeec06a4c0 msgr2=0x7efeec06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 -- 172.21.15.111:0/49012126 shutdown_connections 2023-12-04T05:40:06.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.569+0000 7efef411d640 1 -- 172.21.15.111:0/49012126 wait complete. 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.570+0000 7efef411d640 1 Processor -- start 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.570+0000 7efef411d640 1 -- start start 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8003970 0x7efeec071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efeec138160 0x7efeec1379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 0x7efeec158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec158f80 con 0x7efeec1585c0 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1389e0 con 0x7efed8003970 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec139100 con 0x7efeec138160 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 0x7efeec158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 0x7efeec158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41110/0 (socket says 172.21.15.111:41110) 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 -- 172.21.15.111:0/3177560590 learned_addr learned my addr 172.21.15.111:0/3177560590 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef1eb9640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8003970 0x7efeec071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef16b8640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efeec138160 0x7efeec1379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 -- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efeec138160 msgr2=0x7efeec1379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efeec138160 0x7efeec1379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 -- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8003970 msgr2=0x7efeec071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8003970 0x7efeec071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 -- 172.21.15.111:0/3177560590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec137f10 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef26ba640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 0x7efeec158a40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efeec1405e0 tx=0x7efee4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400f620 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400f780 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec072220 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400faf0 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.571+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec139630 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.572+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec0693d0 con 0x7efeec1585c0 2023-12-04T05:40:06.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.573+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7efee400fc50 con 0x7efeec1585c0 2023-12-04T05:40:06.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.573+0000 7efee27fc640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7efed006b2d0 0x7efed006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.573+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efee40895d0 con 0x7efeec1585c0 2023-12-04T05:40:06.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.573+0000 7efef1eb9640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7efed006b2d0 0x7efed006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.574+0000 7efef1eb9640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7efed006b2d0 0x7efed006d790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efee8034000 tx=0x7efee8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.577+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efee4054820 con 0x7efeec1585c0 2023-12-04T05:40:06.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.809+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7efeec06d170 con 0x7efeec1585c0 2023-12-04T05:40:06.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.811+0000 7efee27fc640 1 -- 172.21.15.111:0/3177560590 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7efee4057e40 con 0x7efeec1585c0 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7efed006b2d0 msgr2=0x7efed006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7efed006b2d0 0x7efed006d790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efee8034000 tx=0x7efee8009030 comp rx=0 tx=0).stop 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 msgr2=0x7efeec158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 0x7efeec158a40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efeec1405e0 tx=0x7efee4007430 comp rx=0 tx=0).stop 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 shutdown_connections 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7efed006b2d0 0x7efed006d790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeec1585c0 0x7efeec158a40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efeec138160 0x7efeec1379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 --2- 172.21.15.111:0/3177560590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed8003970 0x7efeec071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 >> 172.21.15.111:0/3177560590 conn(0x7efeec06a4c0 msgr2=0x7efeec15a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:06.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 shutdown_connections 2023-12-04T05:40:06.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.813+0000 7efef411d640 1 -- 172.21.15.111:0/3177560590 wait complete. 2023-12-04T05:40:06.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-04T05:40:06.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 Processor -- start 2023-12-04T05:40:06.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 -- start start 2023-12-04T05:40:06.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f927406b4d0 0x7f927406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92740add00 0x7f92740ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927406be70 con 0x7f92741544c0 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92740ad5d0 con 0x7f92740add00 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.926+0000 7f9278a83640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927406df90 con 0x7f927406b4d0 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9272d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92740add00 0x7f92740ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f927259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f92741544c0 0x7f927406cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47270/0 (socket says 172.21.15.111:47270) 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f927259c640 1 -- 172.21.15.111:0/4166432666 learned_addr learned my addr 172.21.15.111:0/4166432666 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9272d9d640 1 --2- 172.21.15.111:0/4166432666 >> v2:172.21.15.111:3300/0 conn(0x7f92740add00 0x7f92740ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41116/0 (socket says 172.21.15.111:41116) 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f926bfff640 1 --2- 172.21.15.111:0/4166432666 >> v2:172.21.15.192:3300/0 conn(0x7f927406b4d0 0x7f927406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9272d9d640 1 -- 172.21.15.111:0/4166432666 >> v2:172.21.15.192:3300/0 conn(0x7f927406b4d0 msgr2=0x7f927406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9272d9d640 1 --2- 172.21.15.111:0/4166432666 >> v2:172.21.15.192:3300/0 conn(0x7f927406b4d0 0x7f927406b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9272d9d640 1 -- 172.21.15.111:0/4166432666 >> v1:172.21.15.111:6789/0 conn(0x7f92741544c0 legacy=0x7f927406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9272d9d640 1 -- 172.21.15.111:0/4166432666 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92741548a0 con 0x7f92740add00 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f9271d9b640 1 -- 172.21.15.111:0/4166432666 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983651632 0 0) 0x7f927406be70 con 0x7f92741544c0 2023-12-04T05:40:06.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.927+0000 7f926bfff640 1 --2- 172.21.15.111:0/4166432666 >> v2:172.21.15.192:3300/0 conn(0x7f927406b4d0 0x7f927406b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9272d9d640 1 --2- 172.21.15.111:0/4166432666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f92740ae190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9274066460 tx=0x7f926c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b32c40036fb0253 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9271d9b640 1 -- 172.21.15.111:0/4166432666 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c02cc80 con 0x7f92740add00 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9271d9b640 1 -- 172.21.15.111:0/4166432666 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c02cde0 con 0x7f92740add00 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9271d9b640 1 -- 172.21.15.111:0/4166432666 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c034660 con 0x7f92740add00 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 -- 172.21.15.111:0/4166432666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 msgr2=0x7f92740ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 --2- 172.21.15.111:0/4166432666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f92740ae190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9274066460 tx=0x7f926c02bed0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 -- 172.21.15.111:0/4166432666 shutdown_connections 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 --2- 172.21.15.111:0/4166432666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f92740ae190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 --2- 172.21.15.111:0/4166432666 >> v2:172.21.15.192:3300/0 conn(0x7f927406b4d0 0x7f927406b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 -- 172.21.15.111:0/4166432666 >> 172.21.15.111:0/4166432666 conn(0x7f92740b1b50 msgr2=0x7f9274070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:06.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 -- 172.21.15.111:0/4166432666 shutdown_connections 2023-12-04T05:40:06.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.928+0000 7f9278a83640 1 -- 172.21.15.111:0/4166432666 wait complete. 2023-12-04T05:40:06.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 Processor -- start 2023-12-04T05:40:06.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 -- start start 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406b4d0 0x7f92741f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f9274154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92741544c0 0x7f927414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927414bf80 con 0x7f92740add00 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b3ed0 con 0x7f927406b4d0 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f9278a83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b41b0 con 0x7f92741544c0 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f926bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f9274154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f926bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f9274154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41130/0 (socket says 172.21.15.111:41130) 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.930+0000 7f926bfff640 1 -- 172.21.15.111:0/4240034609 learned_addr learned my addr 172.21.15.111:0/4240034609 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f9272d9d640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92741544c0 0x7f927414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f927259c640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406b4d0 0x7f92741f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926bfff640 1 -- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92741544c0 msgr2=0x7f927414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926bfff640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92741544c0 0x7f927414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926bfff640 1 -- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406b4d0 msgr2=0x7f92741f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926bfff640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406b4d0 0x7f92741f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926bfff640 1 -- 172.21.15.111:0/4240034609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c009a70 con 0x7f92740add00 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f927259c640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406b4d0 0x7f92741f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926bfff640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f9274154a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c7f0 tx=0x7f925c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c007a30 con 0x7f92740add00 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c007b90 con 0x7f92740add00 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92740b4490 con 0x7f92740add00 2023-12-04T05:40:06.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00ce40 con 0x7f92740add00 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.931+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9274154fd0 con 0x7f92740add00 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.932+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927414c130 con 0x7f92740add00 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.933+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f925c0113f0 con 0x7f92740add00 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.933+0000 7f926affd640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925406b2a0 0x7f925406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.933+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f925c0899d0 con 0x7f92740add00 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.933+0000 7f927259c640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925406b2a0 0x7f925406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:06.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.933+0000 7f927259c640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925406b2a0 0x7f925406d760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9264000f60 tx=0x7f9264009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:06.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:06.937+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f927414c130 con 0x7f92740add00 2023-12-04T05:40:07.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.171+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f92740693d0 con 0x7f92740add00 2023-12-04T05:40:07.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.173+0000 7f926affd640 1 -- 172.21.15.111:0/4240034609 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f925c01f0a0 con 0x7f92740add00 2023-12-04T05:40:07.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925406b2a0 msgr2=0x7f925406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925406b2a0 0x7f925406d760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9264000f60 tx=0x7f9264009040 comp rx=0 tx=0).stop 2023-12-04T05:40:07.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 msgr2=0x7f9274154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f9274154a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c7f0 tx=0x7f925c00cbf0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 shutdown_connections 2023-12-04T05:40:07.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925406b2a0 0x7f925406d760 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92741544c0 0x7f927414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92740add00 0x7f9274154a90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 --2- 172.21.15.111:0/4240034609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406b4d0 0x7f92741f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 >> 172.21.15.111:0/4240034609 conn(0x7f92740b1b50 msgr2=0x7f92740af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 shutdown_connections 2023-12-04T05:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.176+0000 7f9278a83640 1 -- 172.21.15.111:0/4240034609 wait complete. 2023-12-04T05:40:07.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-04T05:40:07.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.289+0000 7f6752990640 1 Processor -- start 2023-12-04T05:40:07.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f6752990640 1 -- start start 2023-12-04T05:40:07.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f6752990640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f674c0d1780 0x7f674c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f6752990640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 0x7f674c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f6752990640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c0cdcb0 con 0x7f674c0b20a0 2023-12-04T05:40:07.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f6752990640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c0d39e0 con 0x7f674c0d1780 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f6752990640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0d3cc0 con 0x7f674c0b2ad0 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f674b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f674c0b20a0 0x7f674c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47280/0 (socket says 172.21.15.111:47280) 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.290+0000 7f674b7fe640 1 -- 172.21.15.111:0/3790079484 learned_addr learned my addr 172.21.15.111:0/3790079484 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873236652 0 0) 0x7f674c0cdcb0 con 0x7f674c0b20a0 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6734003520 con 0x7f674c0b20a0 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 --2- 172.21.15.111:0/3790079484 >> v2:172.21.15.192:3300/0 conn(0x7f674c0d1780 0x7f674c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f6750f2d640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 0x7f674c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858799808 0 0) 0x7f6734003520 con 0x7f674c0b20a0 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 msgr2=0x7f674c0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 0x7f674c0ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 >> v2:172.21.15.192:3300/0 conn(0x7f674c0d1780 msgr2=0x7f674c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 --2- 172.21.15.111:0/3790079484 >> v2:172.21.15.192:3300/0 conn(0x7f674c0d1780 0x7f674c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0cdf50 con 0x7f674c0b20a0 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f673c001670 con 0x7f674c0b20a0 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 >> v1:172.21.15.111:6789/0 conn(0x7f674c0b20a0 legacy=0x7f674c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6734003970 0x7f6734003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 0x7f67340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6734004d00 0x7f6734005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0d39e0 con 0x7f6734003970 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0d3cc0 con 0x7f6734004d00 2023-12-04T05:40:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673400b980 con 0x7f674c0b2ad0 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6734003970 0x7f6734003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674b7fe640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 0x7f67340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f6750f2d640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6734004d00 0x7f6734005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 -- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 msgr2=0x7f67340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f674c0b2ad0 0x7f67340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 -- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6734004d00 msgr2=0x7f6734005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6734004d00 0x7f6734005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.291+0000 7f674bfff640 1 -- 172.21.15.111:0/3790079484 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6734003520 con 0x7f6734003970 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.292+0000 7f674bfff640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6734003970 0x7f6734003d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6740009ac0 tx=0x7f67340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8229e7c3145295 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.292+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674002c710 con 0x7f6734003970 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.292+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674002c870 con 0x7f6734003970 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.292+0000 7f674affd640 1 -- 172.21.15.111:0/3790079484 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674002cc00 con 0x7f6734003970 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6752990640 1 -- 172.21.15.111:0/3790079484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6734003970 msgr2=0x7f6734003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6752990640 1 --2- 172.21.15.111:0/3790079484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6734003970 0x7f6734003d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6740009ac0 tx=0x7f67340057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6750f2d640 1 -- 172.21.15.111:0/3790079484 reap_dead start 2023-12-04T05:40:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6752990640 1 -- 172.21.15.111:0/3790079484 shutdown_connections 2023-12-04T05:40:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6752990640 1 -- 172.21.15.111:0/3790079484 >> 172.21.15.111:0/3790079484 conn(0x7f674c09c4e0 msgr2=0x7f674c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6752990640 1 -- 172.21.15.111:0/3790079484 shutdown_connections 2023-12-04T05:40:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.293+0000 7f6752990640 1 -- 172.21.15.111:0/3790079484 wait complete. 2023-12-04T05:40:07.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.294+0000 7f6752990640 1 Processor -- start 2023-12-04T05:40:07.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.294+0000 7f6752990640 1 -- start start 2023-12-04T05:40:07.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6752990640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6734003970 0x7f674c158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6752990640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 0x7f674c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6752990640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f674c0b11f0 0x7f674c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6752990640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0b1bb0 con 0x7f674c1592c0 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6752990640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0b6000 con 0x7f674c0b11f0 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6752990640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c15f450 con 0x7f6734003970 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 0x7f674c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 0x7f674c159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41150/0 (socket says 172.21.15.111:41150) 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 -- 172.21.15.111:0/3448952716 learned_addr learned my addr 172.21.15.111:0/3448952716 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674bfff640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6734003970 0x7f674c158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f6750f2d640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f674c0b11f0 0x7f674c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 -- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6734003970 msgr2=0x7f674c158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6734003970 0x7f674c158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 -- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f674c0b11f0 msgr2=0x7f674c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f674c0b11f0 0x7f674c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 -- 172.21.15.111:0/3448952716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c15e3b0 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674bfff640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6734003970 0x7f674c158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.295+0000 7f674b7fe640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 0x7f674c159740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f673c00e820 tx=0x7f673c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.296+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00c870 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.296+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00c9d0 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.296+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c15e690 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.296+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00cd80 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.296+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c15ec50 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.297+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c15e100 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.297+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f673c01d460 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.297+0000 7f6748ff9640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f673806b2d0 0x7f673806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.298+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f673c089850 con 0x7f674c1592c0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.298+0000 7f674bfff640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f673806b2d0 0x7f673806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.298+0000 7f674bfff640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f673806b2d0 0x7f673806d790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6740036000 tx=0x7f6740030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:07.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.302+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f673c054a10 con 0x7f674c1592c0 2023-12-04T05:40:07.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.569+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f674c0693d0 con 0x7f674c1592c0 2023-12-04T05:40:07.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.571+0000 7f6748ff9640 1 -- 172.21.15.111:0/3448952716 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f673c058030 con 0x7f674c1592c0 2023-12-04T05:40:07.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f673806b2d0 msgr2=0x7f673806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f673806b2d0 0x7f673806d790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6740036000 tx=0x7f6740030000 comp rx=0 tx=0).stop 2023-12-04T05:40:07.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 msgr2=0x7f674c159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 0x7f674c159740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f673c00e820 tx=0x7f673c00ece0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 shutdown_connections 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f673806b2d0 0x7f673806d790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f674c0b11f0 0x7f674c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f674c1592c0 0x7f674c159740 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 --2- 172.21.15.111:0/3448952716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6734003970 0x7f674c158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 >> 172.21.15.111:0/3448952716 conn(0x7f674c09c4e0 msgr2=0x7f674c06c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 shutdown_connections 2023-12-04T05:40:07.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.573+0000 7f6752990640 1 -- 172.21.15.111:0/3448952716 wait complete. 2023-12-04T05:40:07.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-04T05:40:07.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.686+0000 7fb38a961640 1 Processor -- start 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.686+0000 7fb38a961640 1 -- start start 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.686+0000 7fb38a961640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb3840d1780 0x7fb3840d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.686+0000 7fb38a961640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3840cdcb0 con 0x7fb3840b2ad0 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.686+0000 7fb38a961640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3840d39e0 con 0x7fb3840b20a0 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.686+0000 7fb38a961640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3840d3cc0 con 0x7fb3840d1780 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb388efe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb3840b2ad0 0x7fb3840ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47294/0 (socket says 172.21.15.111:47294) 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb388efe640 1 -- 172.21.15.111:0/4100715469 learned_addr learned my addr 172.21.15.111:0/4100715469 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836324075 0 0) 0x7fb3840cdcb0 con 0x7fb3840b2ad0 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb383fff640 1 --2- 172.21.15.111:0/4100715469 >> v2:172.21.15.192:3300/0 conn(0x7fb3840d1780 0x7fb3840d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb36c003520 con 0x7fb3840b2ad0 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102916042 0 0) 0x7fb36c003520 con 0x7fb3840b2ad0 2023-12-04T05:40:07.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 >> v2:172.21.15.192:3300/0 conn(0x7fb3840d1780 msgr2=0x7fb3840d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 --2- 172.21.15.111:0/4100715469 >> v2:172.21.15.192:3300/0 conn(0x7fb3840d1780 0x7fb3840d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 >> v1:172.21.15.192:6789/0 conn(0x7fb3840b20a0 legacy=0x7fb3840d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840cdf50 con 0x7fb3840b2ad0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fb37c0019f0 con 0x7fb3840b2ad0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 >> v1:172.21.15.111:6789/0 conn(0x7fb3840b2ad0 legacy=0x7fb3840ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36c0039c0 0x7fb36c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb36c0047c0 0x7fb36c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0052a0 0x7fb36c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840d3cc0 con 0x7fb36c0047c0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c00c1c0 con 0x7fb36c0052a0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c00c380 con 0x7fb36c0039c0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb3837fe640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb36c0047c0 0x7fb36c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb388efe640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0052a0 0x7fb36c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.687+0000 7fb383fff640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36c0039c0 0x7fb36c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb3837fe640 1 -- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36c0039c0 msgr2=0x7fb36c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb3837fe640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36c0039c0 0x7fb36c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb3837fe640 1 -- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0052a0 msgr2=0x7fb36c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb3837fe640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0052a0 0x7fb36c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb3837fe640 1 -- 172.21.15.111:0/4100715469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c003520 con 0x7fb36c0047c0 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb388efe640 1 -- 172.21.15.111:0/4100715469 reap_dead start 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb383fff640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36c0039c0 0x7fb36c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb3837fe640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb36c0047c0 0x7fb36c004c40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb378001750 tx=0x7fb36c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42566038e0360479 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780052d0 con 0x7fb36c0047c0 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37802e050 con 0x7fb36c0047c0 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.688+0000 7fb382ffd640 1 -- 172.21.15.111:0/4100715469 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378003830 con 0x7fb36c0047c0 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 -- 172.21.15.111:0/4100715469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb36c0047c0 msgr2=0x7fb36c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb36c0047c0 0x7fb36c004c40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb378001750 tx=0x7fb36c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 -- 172.21.15.111:0/4100715469 shutdown_connections 2023-12-04T05:40:07.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 --2- 172.21.15.111:0/4100715469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb36c0047c0 0x7fb36c004c40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 -- 172.21.15.111:0/4100715469 >> 172.21.15.111:0/4100715469 conn(0x7fb38409c4e0 msgr2=0x7fb3840d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:07.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 -- 172.21.15.111:0/4100715469 shutdown_connections 2023-12-04T05:40:07.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.689+0000 7fb38a961640 1 -- 172.21.15.111:0/4100715469 wait complete. 2023-12-04T05:40:07.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.690+0000 7fb38a961640 1 Processor -- start 2023-12-04T05:40:07.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 -- start start 2023-12-04T05:40:07.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0047c0 0x7fb3840cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 0x7fb3841f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3840c8ea0 0x7fb3840c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840b8ca0 con 0x7fb3841f2f30 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3841f3ea0 con 0x7fb36c0047c0 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb38a961640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840c9860 con 0x7fb3840c8ea0 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb3837fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 0x7fb3841f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb3837fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 0x7fb3841f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41170/0 (socket says 172.21.15.111:41170) 2023-12-04T05:40:07.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb3837fe640 1 -- 172.21.15.111:0/250478042 learned_addr learned my addr 172.21.15.111:0/250478042 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb383fff640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0047c0 0x7fb3840cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.691+0000 7fb388efe640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3840c8ea0 0x7fb3840c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb3837fe640 1 -- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3840c8ea0 msgr2=0x7fb3840c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb3837fe640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3840c8ea0 0x7fb3840c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb3837fe640 1 -- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0047c0 msgr2=0x7fb3840cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb3837fe640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0047c0 0x7fb3840cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb3837fe640 1 -- 172.21.15.111:0/250478042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3841f3bc0 con 0x7fb3841f2f30 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb3837fe640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 0x7fb3841f3390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb378000c00 tx=0x7fb378001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780076c0 con 0x7fb3841f2f30 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb378007a30 con 0x7fb3841f2f30 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3841f0ce0 con 0x7fb3841f2f30 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378006be0 con 0x7fb3841f2f30 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.692+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3841f12a0 con 0x7fb3841f2f30 2023-12-04T05:40:07.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.693+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3840693d0 con 0x7fb3841f2f30 2023-12-04T05:40:07.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.693+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb378006d40 con 0x7fb3841f2f30 2023-12-04T05:40:07.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.694+0000 7fb380ff9640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb37006b2a0 0x7fb37006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:07.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.694+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb3780ab2a0 con 0x7fb3841f2f30 2023-12-04T05:40:07.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.694+0000 7fb383fff640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb37006b2a0 0x7fb37006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:07.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.694+0000 7fb383fff640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb37006b2a0 0x7fb37006d760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb384063ea0 tx=0x7fb374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:07.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.698+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3780765a0 con 0x7fb3841f2f30 2023-12-04T05:40:07.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.949+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fb3840d40b0 con 0x7fb3841f2f30 2023-12-04T05:40:07.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.951+0000 7fb380ff9640 1 -- 172.21.15.111:0/250478042 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb378079bc0 con 0x7fb3841f2f30 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb37006b2a0 msgr2=0x7fb37006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb37006b2a0 0x7fb37006d760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb384063ea0 tx=0x7fb374008040 comp rx=0 tx=0).stop 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 msgr2=0x7fb3841f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 0x7fb3841f3390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb378000c00 tx=0x7fb378001120 comp rx=0 tx=0).stop 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 shutdown_connections 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb37006b2a0 0x7fb37006d760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3840c8ea0 0x7fb3840c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3841f2f30 0x7fb3841f3390 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 --2- 172.21.15.111:0/250478042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb36c0047c0 0x7fb3840cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 >> 172.21.15.111:0/250478042 conn(0x7fb38409c4e0 msgr2=0x7fb3840ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 shutdown_connections 2023-12-04T05:40:07.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:07.954+0000 7fb38a961640 1 -- 172.21.15.111:0/250478042 wait complete. 2023-12-04T05:40:07.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:08.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.064+0000 7f38c2ffc640 1 Processor -- start 2023-12-04T05:40:08.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.065+0000 7f38c2ffc640 1 -- start start 2023-12-04T05:40:08.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.065+0000 7f38c2ffc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f38bc15f4e0 0x7f38bc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.065+0000 7f38c2ffc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 0x7f38bc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.065+0000 7f38c2ffc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc0cbca0 con 0x7f38bc15b480 2023-12-04T05:40:08.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.065+0000 7f38c2ffc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0cbf40 con 0x7f38bc15f4e0 2023-12-04T05:40:08.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.065+0000 7f38c2ffc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0cc220 con 0x7f38bc0cf9f0 2023-12-04T05:40:08.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 0x7f38bc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38bbfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f38bc15f4e0 0x7f38bc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 0x7f38bc0cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:41756/0 (socket says 172.21.15.111:41756) 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 -- 172.21.15.111:0/1222206632 learned_addr learned my addr 172.21.15.111:0/1222206632 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c0d98640 1 --1- 172.21.15.111:0/1222206632 >> v1:172.21.15.192:6789/0 conn(0x7f38bc15b480 0x7f38bc15edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:56868/0 (socket says 172.21.15.111:56868) 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 -- 172.21.15.111:0/1222206632 >> v2:172.21.15.192:3300/0 conn(0x7f38bc15f4e0 msgr2=0x7f38bc0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 --2- 172.21.15.111:0/1222206632 >> v2:172.21.15.192:3300/0 conn(0x7f38bc15f4e0 0x7f38bc0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 -- 172.21.15.111:0/1222206632 >> v1:172.21.15.192:6789/0 conn(0x7f38bc15b480 legacy=0x7f38bc15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.066+0000 7f38c1599640 1 -- 172.21.15.111:0/1222206632 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0cc500 con 0x7f38bc0cf9f0 2023-12-04T05:40:08.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.067+0000 7f38c1599640 1 --2- 172.21.15.111:0/1222206632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 0x7f38bc0cca20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009e50 tx=0x7f38b402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5f5139dc1e4f1a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.067+0000 7f38bb7fe640 1 -- 172.21.15.111:0/1222206632 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b402cb00 con 0x7f38bc0cf9f0 2023-12-04T05:40:08.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.067+0000 7f38bb7fe640 1 -- 172.21.15.111:0/1222206632 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b402cc60 con 0x7f38bc0cf9f0 2023-12-04T05:40:08.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.067+0000 7f38bb7fe640 1 -- 172.21.15.111:0/1222206632 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b40343f0 con 0x7f38bc0cf9f0 2023-12-04T05:40:08.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 -- 172.21.15.111:0/1222206632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 msgr2=0x7f38bc0cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/1222206632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 0x7f38bc0cca20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009e50 tx=0x7f38b402bf60 comp rx=0 tx=0).stop 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 -- 172.21.15.111:0/1222206632 shutdown_connections 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/1222206632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc0cf9f0 0x7f38bc0cca20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/1222206632 >> v2:172.21.15.192:3300/0 conn(0x7f38bc15f4e0 0x7f38bc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 -- 172.21.15.111:0/1222206632 >> 172.21.15.111:0/1222206632 conn(0x7f38bc0b2a70 msgr2=0x7f38bc0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 -- 172.21.15.111:0/1222206632 shutdown_connections 2023-12-04T05:40:08.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.068+0000 7f38c2ffc640 1 -- 172.21.15.111:0/1222206632 wait complete. 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.069+0000 7f38c2ffc640 1 Processor -- start 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.069+0000 7f38c2ffc640 1 -- start start 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38bc0cf9f0 0x7f38bc153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc15b480 0x7f38bc1e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 0x7f38bc06f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c5f60 con 0x7f38bc15f4e0 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc1541d0 con 0x7f38bc0cf9f0 2023-12-04T05:40:08.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc1e5f40 con 0x7f38bc15b480 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 0x7f38bc06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 0x7f38bc06f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41184/0 (socket says 172.21.15.111:41184) 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 -- 172.21.15.111:0/3065821703 learned_addr learned my addr 172.21.15.111:0/3065821703 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38bbfff640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc15b480 0x7f38bc1e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c0d98640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38bc0cf9f0 0x7f38bc153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 -- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc15b480 msgr2=0x7f38bc1e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc15b480 0x7f38bc1e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 -- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38bc0cf9f0 msgr2=0x7f38bc153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38bc0cf9f0 0x7f38bc153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 -- 172.21.15.111:0/3065821703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009b30 con 0x7f38bc15f4e0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c1599640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 0x7f38bc06f540 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009e50 tx=0x7f38b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4009510 con 0x7f38bc15f4e0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b403e450 con 0x7f38bc15f4e0 2023-12-04T05:40:08.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc1e6220 con 0x7f38bc15f4e0 2023-12-04T05:40:08.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.070+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b403e800 con 0x7f38bc15f4e0 2023-12-04T05:40:08.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.071+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc0c6420 con 0x7f38bc15f4e0 2023-12-04T05:40:08.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.071+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc0c6610 con 0x7f38bc15f4e0 2023-12-04T05:40:08.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.072+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f38b403e960 con 0x7f38bc15f4e0 2023-12-04T05:40:08.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.072+0000 7f38b97fa640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f38a006b2a0 0x7f38a006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.072+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38b40ac630 con 0x7f38bc15f4e0 2023-12-04T05:40:08.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.072+0000 7f38c0d98640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f38a006b2a0 0x7f38a006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.073+0000 7f38c0d98640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f38a006b2a0 0x7f38a006d760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000f60 tx=0x7f38b0007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.080+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38b4077760 con 0x7f38bc15f4e0 2023-12-04T05:40:08.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.254+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38bc067460 con 0x7f38bc15f4e0 2023-12-04T05:40:08.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.255+0000 7f38b97fa640 1 -- 172.21.15.111:0/3065821703 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f38b407ad80 con 0x7f38bc15f4e0 2023-12-04T05:40:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:08.257 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":29,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:05.375140+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:08.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f38a006b2a0 msgr2=0x7f38a006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f38a006b2a0 0x7f38a006d760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000f60 tx=0x7f38b0007c80 comp rx=0 tx=0).stop 2023-12-04T05:40:08.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 msgr2=0x7f38bc06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 0x7f38bc06f540 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009e50 tx=0x7f38b4004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 shutdown_connections 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f38a006b2a0 0x7f38a006d760 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38bc15f4e0 0x7f38bc06f540 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38bc15b480 0x7f38bc1e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 --2- 172.21.15.111:0/3065821703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38bc0cf9f0 0x7f38bc153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 >> 172.21.15.111:0/3065821703 conn(0x7f38bc0b2a70 msgr2=0x7f38bc0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 shutdown_connections 2023-12-04T05:40:08.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.257+0000 7f38c2ffc640 1 -- 172.21.15.111:0/3065821703 wait complete. 2023-12-04T05:40:08.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:08.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:08.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.371+0000 7f4f0f031640 1 Processor -- start 2023-12-04T05:40:08.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.371+0000 7f4f0f031640 1 -- start start 2023-12-04T05:40:08.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0f031640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4f0815b0c0 0x7f4f0815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0f031640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4f080ce940 0x7f4f080cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0f031640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f08071b40 0x7f4f08071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0f031640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f080724f0 con 0x7f4f080ce940 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0f031640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f080cd320 con 0x7f4f0815b0c0 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0f031640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080ced80 con 0x7f4f08071b40 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f07fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4f080ce940 0x7f4f080cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f07fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4f080ce940 0x7f4f080cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41200/0 (socket says 172.21.15.111:41200) 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f07fff640 1 -- 172.21.15.111:0/545700346 learned_addr learned my addr 172.21.15.111:0/545700346 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0cdcd640 1 --2- 172.21.15.111:0/545700346 >> v2:172.21.15.192:3300/0 conn(0x7f4f0815b0c0 0x7f4f0815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.372+0000 7f4f0d5ce640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f08071b40 0x7f4f08071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f07fff640 1 -- 172.21.15.111:0/545700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f08071b40 msgr2=0x7f4f08071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f07fff640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f08071b40 0x7f4f08071fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f07fff640 1 -- 172.21.15.111:0/545700346 >> v2:172.21.15.192:3300/0 conn(0x7f4f0815b0c0 msgr2=0x7f4f0815ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f07fff640 1 --2- 172.21.15.111:0/545700346 >> v2:172.21.15.192:3300/0 conn(0x7f4f0815b0c0 0x7f4f0815ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f07fff640 1 -- 172.21.15.111:0/545700346 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f080cbee0 con 0x7f4f080ce940 2023-12-04T05:40:08.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f0d5ce640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f08071b40 0x7f4f08071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:08.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.373+0000 7f4f07fff640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f080ce940 0x7f4f080cb9a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4f0806b470 tx=0x7f4ef402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ee3c13a19d1645 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.374+0000 7f4f077fe640 1 -- 172.21.15.111:0/545700346 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40317e0 con 0x7f4f080ce940 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.374+0000 7f4f077fe640 1 -- 172.21.15.111:0/545700346 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef4031940 con 0x7f4f080ce940 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.374+0000 7f4f077fe640 1 -- 172.21.15.111:0/545700346 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4031cd0 con 0x7f4f080ce940 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 -- 172.21.15.111:0/545700346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f080ce940 msgr2=0x7f4f080cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f080ce940 0x7f4f080cb9a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4f0806b470 tx=0x7f4ef402b880 comp rx=0 tx=0).stop 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 -- 172.21.15.111:0/545700346 shutdown_connections 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f08071b40 0x7f4f08071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 --2- 172.21.15.111:0/545700346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f080ce940 0x7f4f080cb9a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 --2- 172.21.15.111:0/545700346 >> v2:172.21.15.192:3300/0 conn(0x7f4f0815b0c0 0x7f4f0815ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 -- 172.21.15.111:0/545700346 >> 172.21.15.111:0/545700346 conn(0x7f4f080b0cf0 msgr2=0x7f4f080b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 -- 172.21.15.111:0/545700346 shutdown_connections 2023-12-04T05:40:08.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.375+0000 7f4f0f031640 1 -- 172.21.15.111:0/545700346 wait complete. 2023-12-04T05:40:08.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.376+0000 7f4f0f031640 1 Processor -- start 2023-12-04T05:40:08.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.376+0000 7f4f0f031640 1 -- start start 2023-12-04T05:40:08.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0f031640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f08071b40 0x7f4f08148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0f031640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f080ce940 0x7f4f080a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0f031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 0x7f4f08158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0f031640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08157ef0 con 0x7f4f0815b0c0 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0f031640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081572e0 con 0x7f4f08071b40 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0f031640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081575c0 con 0x7f4f080ce940 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 0x7f4f08158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 0x7f4f08158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41216/0 (socket says 172.21.15.111:41216) 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 -- 172.21.15.111:0/4210835786 learned_addr learned my addr 172.21.15.111:0/4210835786 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0cdcd640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f08071b40 0x7f4f08148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f07fff640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f080ce940 0x7f4f080a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 -- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f080ce940 msgr2=0x7f4f080a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f080ce940 0x7f4f080a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 -- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f08071b40 msgr2=0x7f4f08148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f08071b40 0x7f4f08148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 -- 172.21.15.111:0/4210835786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef40094d0 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f07fff640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f080ce940 0x7f4f080a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.377+0000 7f4f0d5ce640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 0x7f4f08158380 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f00007ef0 tx=0x7f4f0000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.378+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00003b90 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.378+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0000f040 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.378+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f081578a0 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.378+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000de80 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.378+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f08148d90 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.379+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f08158060 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.379+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4f00004030 con 0x7f4f0815b0c0 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.379+0000 7f4f057fa640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4eec06b2a0 0x7f4eec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.379+0000 7f4f0cdcd640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4eec06b2a0 0x7f4eec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.380+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f00089b20 con 0x7f4f0815b0c0 2023-12-04T05:40:08.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.380+0000 7f4f0cdcd640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4eec06b2a0 0x7f4eec06d760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f0815e290 tx=0x7f4efc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.384+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f00054d90 con 0x7f4f0815b0c0 2023-12-04T05:40:08.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.549+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f08069360 con 0x7f4f0815b0c0 2023-12-04T05:40:08.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.549+0000 7f4f057fa640 1 -- 172.21.15.111:0/4210835786 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f0001a0a0 con 0x7f4f0815b0c0 2023-12-04T05:40:08.551 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4eec06b2a0 msgr2=0x7f4eec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4eec06b2a0 0x7f4eec06d760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f0815e290 tx=0x7f4efc009190 comp rx=0 tx=0).stop 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 msgr2=0x7f4f08158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 0x7f4f08158380 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f00007ef0 tx=0x7f4f0000d4a0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 shutdown_connections 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4eec06b2a0 0x7f4eec06d760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f0815b0c0 0x7f4f08158380 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f080ce940 0x7f4f080a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 --2- 172.21.15.111:0/4210835786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f08071b40 0x7f4f08148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 >> 172.21.15.111:0/4210835786 conn(0x7f4f080b0cf0 msgr2=0x7f4f08154150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 shutdown_connections 2023-12-04T05:40:08.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.551+0000 7f4f0f031640 1 -- 172.21.15.111:0/4210835786 wait complete. 2023-12-04T05:40:08.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-04T05:40:08.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.663+0000 7fb8089c9640 1 Processor -- start 2023-12-04T05:40:08.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.663+0000 7fb8089c9640 1 -- start start 2023-12-04T05:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb8089c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb80406b4d0 0x7fb80406b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb8089c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 0x7fb804151d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb8089c9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040ccfe0 con 0x7fb80415c060 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb8089c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80406bec0 con 0x7fb80406b4d0 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb8089c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040cd240 con 0x7fb804151890 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb80259c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb80415c060 0x7fb8040cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47298/0 (socket says 172.21.15.111:47298) 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.664+0000 7fb80259c640 1 -- 172.21.15.111:0/2022923792 learned_addr learned my addr 172.21.15.111:0/2022923792 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300210805 0 0) 0x7fb8040ccfe0 con 0x7fb80415c060 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f0003520 con 0x7fb80415c060 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb7fbfff640 1 --2- 172.21.15.111:0/2022923792 >> v2:172.21.15.192:3300/0 conn(0x7fb80406b4d0 0x7fb80406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 0x7fb804151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546405613 0 0) 0x7fb7f0003520 con 0x7fb80415c060 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 msgr2=0x7fb804151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 0x7fb804151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 >> v2:172.21.15.192:3300/0 conn(0x7fb80406b4d0 msgr2=0x7fb80406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 --2- 172.21.15.111:0/2022923792 >> v2:172.21.15.192:3300/0 conn(0x7fb80406b4d0 0x7fb80406b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040cd520 con 0x7fb80415c060 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fb7f4001670 con 0x7fb80415c060 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 >> v1:172.21.15.111:6789/0 conn(0x7fb80415c060 legacy=0x7fb8040cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 0x7fb7f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7f0003ef0 0x7fb7f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80406bec0 con 0x7fb7f0004d00 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040cd240 con 0x7fb7f0003ef0 2023-12-04T05:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f000b9d0 con 0x7fb804151890 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb80259c640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 0x7fb7f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb7fbfff640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7f0003ef0 0x7fb7f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 -- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 msgr2=0x7fb7f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804151890 0x7fb7f0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 -- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7f0003ef0 msgr2=0x7fb7f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7f0003ef0 0x7fb7f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.665+0000 7fb802d9d640 1 -- 172.21.15.111:0/2022923792 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0003520 con 0x7fb7f0004d00 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.666+0000 7fb802d9d640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e170 tx=0x7fb7f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa3dcf62d727d55 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.666+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02cbf0 con 0x7fb7f0004d00 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.666+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc00f040 con 0x7fb7f0004d00 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.666+0000 7fb801d9b640 1 -- 172.21.15.111:0/2022923792 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc034440 con 0x7fb7f0004d00 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb8089c9640 1 -- 172.21.15.111:0/2022923792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f0004d00 msgr2=0x7fb7f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb8089c9640 1 --2- 172.21.15.111:0/2022923792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7f0004d00 0x7fb7f0005180 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e170 tx=0x7fb7f00057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb802d9d640 1 -- 172.21.15.111:0/2022923792 reap_dead start 2023-12-04T05:40:08.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb8089c9640 1 -- 172.21.15.111:0/2022923792 shutdown_connections 2023-12-04T05:40:08.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb8089c9640 1 -- 172.21.15.111:0/2022923792 >> 172.21.15.111:0/2022923792 conn(0x7fb8040b1b50 msgr2=0x7fb804070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb8089c9640 1 -- 172.21.15.111:0/2022923792 shutdown_connections 2023-12-04T05:40:08.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.667+0000 7fb8089c9640 1 -- 172.21.15.111:0/2022923792 wait complete. 2023-12-04T05:40:08.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.668+0000 7fb8089c9640 1 Processor -- start 2023-12-04T05:40:08.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.668+0000 7fb8089c9640 1 -- start start 2023-12-04T05:40:08.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 0x7fb804147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804146420 0x7fb80415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb80415dcd0 0x7fb8040ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804149d30 con 0x7fb804145c70 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804147a80 con 0x7fb80415dcd0 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415e150 con 0x7fb804146420 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 0x7fb804147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 0x7fb804147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41242/0 (socket says 172.21.15.111:41242) 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 -- 172.21.15.111:0/923029209 learned_addr learned my addr 172.21.15.111:0/923029209 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb802d9d640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb80415dcd0 0x7fb8040ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb7fbfff640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804146420 0x7fb80415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 -- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804146420 msgr2=0x7fb80415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804146420 0x7fb80415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 -- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb80415dcd0 msgr2=0x7fb8040ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb80415dcd0 0x7fb8040ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 -- 172.21.15.111:0/923029209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804149130 con 0x7fb804145c70 2023-12-04T05:40:08.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb7fbfff640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804146420 0x7fb80415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:08.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb802d9d640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb80415dcd0 0x7fb8040ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:08.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb80259c640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 0x7fb804147540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4007310 tx=0x7fb7f40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400fa20 con 0x7fb804145c70 2023-12-04T05:40:08.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f400fba0 con 0x7fb804145c70 2023-12-04T05:40:08.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8041493d0 con 0x7fb804145c70 2023-12-04T05:40:08.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40133f0 con 0x7fb804145c70 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.669+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb804149670 con 0x7fb804145c70 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.671+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb804066950 con 0x7fb804145c70 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.671+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb7f4013550 con 0x7fb804145c70 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.671+0000 7fb7faffd640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb7e406b2a0 0x7fb7e406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.671+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb7f4089bc0 con 0x7fb804145c70 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.671+0000 7fb7fbfff640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb7e406b2a0 0x7fb7e406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.672+0000 7fb7fbfff640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb7e406b2a0 0x7fb7e406d760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8040b2a40 tx=0x7fb7ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.675+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7f4054d80 con 0x7fb804145c70 2023-12-04T05:40:08.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.843+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb8040693d0 con 0x7fb804145c70 2023-12-04T05:40:08.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.843+0000 7fb7faffd640 1 -- 172.21.15.111:0/923029209 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb7f40583a0 con 0x7fb804145c70 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:08.845 INFO:teuthology.orchestra.run.smithi111.stdout:] 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb7e406b2a0 msgr2=0x7fb7e406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb7e406b2a0 0x7fb7e406d760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8040b2a40 tx=0x7fb7ec008040 comp rx=0 tx=0).stop 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 msgr2=0x7fb804147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 0x7fb804147540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4007310 tx=0x7fb7f40077d0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 shutdown_connections 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb7e406b2a0 0x7fb7e406d760 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb80415dcd0 0x7fb8040ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb804146420 0x7fb80415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 --2- 172.21.15.111:0/923029209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb804145c70 0x7fb804147540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 >> 172.21.15.111:0/923029209 conn(0x7fb8040b1b50 msgr2=0x7fb804070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 shutdown_connections 2023-12-04T05:40:08.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.845+0000 7fb8089c9640 1 -- 172.21.15.111:0/923029209 wait complete. 2023-12-04T05:40:08.857 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-04T05:40:08.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-04T05:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.961+0000 7f2e4003d640 1 Processor -- start 2023-12-04T05:40:08.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.961+0000 7f2e4003d640 1 -- start start 2023-12-04T05:40:08.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e4003d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2e38071a60 0x7f2e3815a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e4003d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e3815dd60 0x7f2e380b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e4003d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3815e1a0 con 0x7f2e380aed40 2023-12-04T05:40:08.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e4003d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e380b48a0 con 0x7f2e38071a60 2023-12-04T05:40:08.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e4003d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380b4b80 con 0x7f2e3815dd60 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e3d5d8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2e38071a60 0x7f2e3815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e3d5d8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2e38071a60 0x7f2e3815a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41244/0 (socket says 172.21.15.111:41244) 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/3321428490 learned_addr learned my addr 172.21.15.111:0/3321428490 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.962+0000 7f2e3e5da640 1 --2- 172.21.15.111:0/3321428490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e3815dd60 0x7f2e380b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.963+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/3321428490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e3815dd60 msgr2=0x7f2e380b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.963+0000 7f2e3d5d8640 1 --2- 172.21.15.111:0/3321428490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e3815dd60 0x7f2e380b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.963+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/3321428490 >> v1:172.21.15.192:6789/0 conn(0x7f2e380aed40 legacy=0x7f2e38071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.963+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/3321428490 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38159d90 con 0x7f2e38071a60 2023-12-04T05:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.963+0000 7f2e3d5d8640 1 --2- 172.21.15.111:0/3321428490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38071a60 0x7f2e3815a870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400bb00 tx=0x7f2e3402df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddda0d8535f6ad25 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.963+0000 7f2e3cdd7640 1 -- 172.21.15.111:0/3321428490 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3402ece0 con 0x7f2e38071a60 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e3cdd7640 1 -- 172.21.15.111:0/3321428490 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34032070 con 0x7f2e38071a60 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e3cdd7640 1 -- 172.21.15.111:0/3321428490 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34038620 con 0x7f2e38071a60 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 -- 172.21.15.111:0/3321428490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38071a60 msgr2=0x7f2e3815a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 --2- 172.21.15.111:0/3321428490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38071a60 0x7f2e3815a870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400bb00 tx=0x7f2e3402df30 comp rx=0 tx=0).stop 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 -- 172.21.15.111:0/3321428490 shutdown_connections 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 --2- 172.21.15.111:0/3321428490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e3815dd60 0x7f2e380b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 --2- 172.21.15.111:0/3321428490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e38071a60 0x7f2e3815a870 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 -- 172.21.15.111:0/3321428490 >> 172.21.15.111:0/3321428490 conn(0x7f2e380b1b90 msgr2=0x7f2e38070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 -- 172.21.15.111:0/3321428490 shutdown_connections 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.964+0000 7f2e4003d640 1 -- 172.21.15.111:0/3321428490 wait complete. 2023-12-04T05:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.965+0000 7f2e4003d640 1 Processor -- start 2023-12-04T05:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.965+0000 7f2e4003d640 1 -- start start 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e4003d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e38071a60 0x7f2e3814d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e4003d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 0x7f2e381555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e4003d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e3815dd60 0x7f2e38155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e4003d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380b11e0 con 0x7f2e380aed40 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e4003d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38153ac0 con 0x7f2e3815dd60 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e4003d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380ad230 con 0x7f2e38071a60 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3d5d8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 0x7f2e381555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3d5d8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 0x7f2e381555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41256/0 (socket says 172.21.15.111:41256) 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/2214860473 learned_addr learned my addr 172.21.15.111:0/2214860473 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3e5da640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e3815dd60 0x7f2e38155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3ddd9640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e38071a60 0x7f2e3814d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e38071a60 msgr2=0x7f2e3814d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3d5d8640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e38071a60 0x7f2e3814d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.966+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e3815dd60 msgr2=0x7f2e38155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e3d5d8640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e3815dd60 0x7f2e38155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e3d5d8640 1 -- 172.21.15.111:0/2214860473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3400b760 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e3e5da640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e3815dd60 0x7f2e38155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e3d5d8640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 0x7f2e381555b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2e3403a040 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34046070 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34038d40 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e380ad4d0 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34042670 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.967+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e38153750 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.968+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e38153c30 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.968+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f2e340427d0 con 0x7f2e380aed40 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.969+0000 7f2e2e7fc640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2e1c06b2a0 0x7f2e1c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:08.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.969+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e340abc40 con 0x7f2e380aed40 2023-12-04T05:40:08.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.969+0000 7f2e3ddd9640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2e1c06b2a0 0x7f2e1c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:08.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.969+0000 7f2e3ddd9640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2e1c06b2a0 0x7f2e1c06d760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007fd0 tx=0x7f2e30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:08.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:08.973+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e34076e90 con 0x7f2e380aed40 2023-12-04T05:40:09.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.141+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2e38069360 con 0x7f2e380aed40 2023-12-04T05:40:09.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.376+0000 7f2e2e7fc640 1 -- 172.21.15.111:0/2214860473 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2e3407a4b0 con 0x7f2e380aed40 2023-12-04T05:40:09.378 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2023-12-04T05:40:09.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.378+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2e1c06b2a0 msgr2=0x7f2e1c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:09.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.378+0000 7f2e4003d640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2e1c06b2a0 0x7f2e1c06d760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007fd0 tx=0x7f2e30009040 comp rx=0 tx=0).stop 2023-12-04T05:40:09.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.378+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 msgr2=0x7f2e381555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:09.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.378+0000 7f2e4003d640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 0x7f2e381555b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2e3403a040 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 shutdown_connections 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f2e1c06b2a0 0x7f2e1c06d760 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e3815dd60 0x7f2e38155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e380aed40 0x7f2e381555b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 --2- 172.21.15.111:0/2214860473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e38071a60 0x7f2e3814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 >> 172.21.15.111:0/2214860473 conn(0x7f2e380b1b90 msgr2=0x7f2e380ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 shutdown_connections 2023-12-04T05:40:09.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.379+0000 7f2e4003d640 1 -- 172.21.15.111:0/2214860473 wait complete. 2023-12-04T05:40:09.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-04T05:40:09.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 Processor -- start 2023-12-04T05:40:09.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 -- start start 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813d170 0x7fb0a813f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0a813fd40 0x7fb0a813d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0a813dcb0 0x7fb0a80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a80cdfd0 con 0x7fb0a813dcb0 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a81402a0 con 0x7fb0a813fd40 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.498+0000 7fb0adbb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8140580 con 0x7fb0a813d170 2023-12-04T05:40:09.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0a813dcb0 0x7fb0a80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0a813dcb0 0x7fb0a80ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41264/0 (socket says 172.21.15.111:41264) 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 -- 172.21.15.111:0/2895459482 learned_addr learned my addr 172.21.15.111:0/2895459482 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a6ffd640 1 --2- 172.21.15.111:0/2895459482 >> v2:172.21.15.192:3300/0 conn(0x7fb0a813fd40 0x7fb0a813d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a77fe640 1 --2- 172.21.15.111:0/2895459482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813d170 0x7fb0a813f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 -- 172.21.15.111:0/2895459482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813d170 msgr2=0x7fb0a813f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 --2- 172.21.15.111:0/2895459482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813d170 0x7fb0a813f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 -- 172.21.15.111:0/2895459482 >> v2:172.21.15.192:3300/0 conn(0x7fb0a813fd40 msgr2=0x7fb0a813d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 --2- 172.21.15.111:0/2895459482 >> v2:172.21.15.192:3300/0 conn(0x7fb0a813fd40 0x7fb0a813d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.499+0000 7fb0a7fff640 1 -- 172.21.15.111:0/2895459482 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80cdc60 con 0x7fb0a813dcb0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0a7fff640 1 --2- 172.21.15.111:0/2895459482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a80ce460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb0a806b8a0 tx=0x7fb0a002e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49c47f3ce01c10c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0a67fc640 1 -- 172.21.15.111:0/2895459482 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a002ea00 con 0x7fb0a813dcb0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0a67fc640 1 -- 172.21.15.111:0/2895459482 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a002eb60 con 0x7fb0a813dcb0 2023-12-04T05:40:09.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0a67fc640 1 -- 172.21.15.111:0/2895459482 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0009600 con 0x7fb0a813dcb0 2023-12-04T05:40:09.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 -- 172.21.15.111:0/2895459482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 msgr2=0x7fb0a80ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:09.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/2895459482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a80ce460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb0a806b8a0 tx=0x7fb0a002e060 comp rx=0 tx=0).stop 2023-12-04T05:40:09.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 -- 172.21.15.111:0/2895459482 shutdown_connections 2023-12-04T05:40:09.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/2895459482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a80ce460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/2895459482 >> v2:172.21.15.192:3300/0 conn(0x7fb0a813fd40 0x7fb0a813d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/2895459482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813d170 0x7fb0a813f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 -- 172.21.15.111:0/2895459482 >> 172.21.15.111:0/2895459482 conn(0x7fb0a80b0cf0 msgr2=0x7fb0a80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.500+0000 7fb0adbb0640 1 -- 172.21.15.111:0/2895459482 shutdown_connections 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.501+0000 7fb0adbb0640 1 -- 172.21.15.111:0/2895459482 wait complete. 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.502+0000 7fb0adbb0640 1 Processor -- start 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.502+0000 7fb0adbb0640 1 -- start start 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.502+0000 7fb0adbb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a813d170 0x7fb0a80b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a813b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813fd40 0x7fb0a80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80b4230 con 0x7fb0a813dcb0 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a813b9e0 con 0x7fb0a813d170 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80b2c50 con 0x7fb0a813fd40 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a813b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a813b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41278/0 (socket says 172.21.15.111:41278) 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 -- 172.21.15.111:0/1095295601 learned_addr learned my addr 172.21.15.111:0/1095295601 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a77fe640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a813d170 0x7fb0a80b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a7fff640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813fd40 0x7fb0a80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 -- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813fd40 msgr2=0x7fb0a80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813fd40 0x7fb0a80b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 -- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a813d170 msgr2=0x7fb0a80b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:09.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a813d170 0x7fb0a80b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 -- 172.21.15.111:0/1095295601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a000bd40 con 0x7fb0a813dcb0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0a6ffd640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a813b4a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e9b0 tx=0x7fb09800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098009670 con 0x7fb0a813dcb0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800c9a0 con 0x7fb0a813dcb0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a80b2f30 con 0x7fb0a813dcb0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800cd50 con 0x7fb0a813dcb0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.503+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a80b34f0 con 0x7fb0a813dcb0 2023-12-04T05:40:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.504+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a8069440 con 0x7fb0a813dcb0 2023-12-04T05:40:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.505+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb098013470 con 0x7fb0a813dcb0 2023-12-04T05:40:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.505+0000 7fb08ffff640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb09406b2a0 0x7fb09406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.505+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb098089ba0 con 0x7fb0a813dcb0 2023-12-04T05:40:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.505+0000 7fb0a77fe640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb09406b2a0 0x7fb09406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.506+0000 7fb0a77fe640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb09406b2a0 0x7fb09406d760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009aa0 tx=0x7fb09c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.510+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb098054ca0 con 0x7fb0a813dcb0 2023-12-04T05:40:09.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:09.680+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb0a8070470 con 0x7fb0a813dcb0 2023-12-04T05:40:10.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.378+0000 7fb08ffff640 1 -- 172.21.15.111:0/1095295601 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb0980170d0 con 0x7fb0a813dcb0 2023-12-04T05:40:10.380 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb09406b2a0 msgr2=0x7fb09406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb09406b2a0 0x7fb09406d760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009aa0 tx=0x7fb09c009190 comp rx=0 tx=0).stop 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 msgr2=0x7fb0a813b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a813b4a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e9b0 tx=0x7fb09800ee70 comp rx=0 tx=0).stop 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 shutdown_connections 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb09406b2a0 0x7fb09406d760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a813fd40 0x7fb0a80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0a813dcb0 0x7fb0a813b4a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 --2- 172.21.15.111:0/1095295601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a813d170 0x7fb0a80b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.381+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 >> 172.21.15.111:0/1095295601 conn(0x7fb0a80b0cf0 msgr2=0x7fb0a80733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:10.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.382+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 shutdown_connections 2023-12-04T05:40:10.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.382+0000 7fb0adbb0640 1 -- 172.21.15.111:0/1095295601 wait complete. 2023-12-04T05:40:10.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-04T05:40:10.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.511+0000 7f9357e2b640 1 Processor -- start 2023-12-04T05:40:10.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.511+0000 7f9357e2b640 1 -- start start 2023-12-04T05:40:10.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9357e2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f9350161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9357e2b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935006eaf0 con 0x7f935015d8a0 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9357e2b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935015e4e0 con 0x7f9350146b10 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9357e2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935015e7c0 con 0x7f93500d1620 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f93563c8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f935015d8a0 0x7f935015dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48750/0 (socket says 172.21.15.111:48750) 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f93563c8640 1 -- 172.21.15.111:0/1453147035 learned_addr learned my addr 172.21.15.111:0/1453147035 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f93553c6640 1 --2- 172.21.15.111:0/1453147035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f9350161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209321822 0 0) 0x7f935006eaf0 con 0x7f935015d8a0 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f933c003520 con 0x7f935015d8a0 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065137575 0 0) 0x7f935015e4e0 con 0x7f9350146b10 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f935006eaf0 con 0x7f9350146b10 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f93553c6640 1 -- 172.21.15.111:0/1453147035 >> v1:172.21.15.192:6789/0 conn(0x7f9350146b10 legacy=0x7f93500d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f93553c6640 1 -- 172.21.15.111:0/1453147035 >> v1:172.21.15.111:6789/0 conn(0x7f935015d8a0 legacy=0x7f935015dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.512+0000 7f93553c6640 1 -- 172.21.15.111:0/1453147035 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350160f50 con 0x7f93500d1620 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.513+0000 7f93553c6640 1 --2- 172.21.15.111:0/1453147035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f9350161a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9340009800 tx=0x7f934002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85bd775a9d128f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.513+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031890 con 0x7f93500d1620 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.513+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400319f0 con 0x7f93500d1620 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.513+0000 7f9354bc5640 1 -- 172.21.15.111:0/1453147035 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340031d60 con 0x7f93500d1620 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 -- 172.21.15.111:0/1453147035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 msgr2=0x7f9350161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 --2- 172.21.15.111:0/1453147035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f9350161a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9340009800 tx=0x7f934002b930 comp rx=0 tx=0).stop 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 -- 172.21.15.111:0/1453147035 shutdown_connections 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 --2- 172.21.15.111:0/1453147035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f9350161a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 -- 172.21.15.111:0/1453147035 >> 172.21.15.111:0/1453147035 conn(0x7f9350070af0 msgr2=0x7f93500662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 -- 172.21.15.111:0/1453147035 shutdown_connections 2023-12-04T05:40:10.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.514+0000 7f9357e2b640 1 -- 172.21.15.111:0/1453147035 wait complete. 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.515+0000 7f9357e2b640 1 Processor -- start 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.515+0000 7f9357e2b640 1 -- start start 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9357e2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f93500b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9357e2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9350146b10 0x7f9350141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9357e2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 0x7f93501418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9357e2b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500ce2e0 con 0x7f935015d8a0 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9357e2b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500cb130 con 0x7f9350146b10 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9357e2b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500ce530 con 0x7f93500d1620 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 0x7f93501418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 0x7f93501418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41990/0 (socket says 172.21.15.111:41990) 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 -- 172.21.15.111:0/2290767155 learned_addr learned my addr 172.21.15.111:0/2290767155 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f9355bc7640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f93500b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:10.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93553c6640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9350146b10 0x7f9350141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 -- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 msgr2=0x7f93500b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f93500b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 -- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9350146b10 msgr2=0x7f9350141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9350146b10 0x7f9350141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 -- 172.21.15.111:0/2290767155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009400 con 0x7f935015d8a0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93563c8640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 0x7f93501418d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f934800caf0 tx=0x7f934800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800aca0 con 0x7f935015d8a0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.516+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800ae00 con 0x7f935015d8a0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.517+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500ce810 con 0x7f935015d8a0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.517+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348012620 con 0x7f935015d8a0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.517+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500b24d0 con 0x7f935015d8a0 2023-12-04T05:40:10.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.517+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93500b26f0 con 0x7f935015d8a0 2023-12-04T05:40:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.518+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9348007070 con 0x7f935015d8a0 2023-12-04T05:40:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.518+0000 7f93467fc640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f933406b2d0 0x7f933406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.518+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f93480899f0 con 0x7f935015d8a0 2023-12-04T05:40:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.518+0000 7f9355bc7640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f933406b2d0 0x7f933406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:10.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.519+0000 7f9355bc7640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f933406b2d0 0x7f933406d790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f934c007fd0 tx=0x7f934c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:10.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.523+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9348054a00 con 0x7f935015d8a0 2023-12-04T05:40:10.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:10.722+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9350069360 con 0x7f935015d8a0 2023-12-04T05:40:11.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.395+0000 7f93467fc640 1 -- 172.21.15.111:0/2290767155 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9348017020 con 0x7f935015d8a0 2023-12-04T05:40:11.397 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 4 and data pool 5 2023-12-04T05:40:11.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.402+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f933406b2d0 msgr2=0x7f933406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.402+0000 7f9357e2b640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f933406b2d0 0x7f933406d790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f934c007fd0 tx=0x7f934c009040 comp rx=0 tx=0).stop 2023-12-04T05:40:11.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.402+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 msgr2=0x7f93501418d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.402+0000 7f9357e2b640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 0x7f93501418d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f934800caf0 tx=0x7f934800a060 comp rx=0 tx=0).stop 2023-12-04T05:40:11.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.403+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 shutdown_connections 2023-12-04T05:40:11.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.403+0000 7f9357e2b640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f933406b2d0 0x7f933406d790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.403+0000 7f9357e2b640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935015d8a0 0x7f93501418d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.403+0000 7f9357e2b640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9350146b10 0x7f9350141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.403+0000 7f9357e2b640 1 --2- 172.21.15.111:0/2290767155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93500d1620 0x7f93500b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.403+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 >> 172.21.15.111:0/2290767155 conn(0x7f9350070af0 msgr2=0x7f935015fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.409+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 shutdown_connections 2023-12-04T05:40:11.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.409+0000 7f9357e2b640 1 -- 172.21.15.111:0/2290767155 wait complete. 2023-12-04T05:40:11.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:11.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 Processor -- start 2023-12-04T05:40:11.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 -- start start 2023-12-04T05:40:11.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 0x7fd6500d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6500b2ad0 0x7fd6500ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6500cdcb0 con 0x7fd6500b20a0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6500d39e0 con 0x7fd6500b2ad0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.532+0000 7fd656d1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500d3cc0 con 0x7fd6500d1780 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd6500b20a0 0x7fd6500d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48758/0 (socket says 172.21.15.111:48758) 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64ffff640 1 -- 172.21.15.111:0/3777788488 learned_addr learned my addr 172.21.15.111:0/3777788488 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd6552ba640 1 --2- 172.21.15.111:0/3777788488 >> v2:172.21.15.192:3300/0 conn(0x7fd6500b2ad0 0x7fd6500ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd654ab9640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 0x7fd6500d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222679278 0 0) 0x7fd6500cdcb0 con 0x7fd6500b20a0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c003520 con 0x7fd6500b20a0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3907573299 0 0) 0x7fd63c003520 con 0x7fd6500b20a0 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 msgr2=0x7fd6500d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 0x7fd6500d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 >> v2:172.21.15.192:3300/0 conn(0x7fd6500b2ad0 msgr2=0x7fd6500ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:11.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 --2- 172.21.15.111:0/3777788488 >> v2:172.21.15.192:3300/0 conn(0x7fd6500b2ad0 0x7fd6500ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6500cdf50 con 0x7fd6500b20a0 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fd638001670 con 0x7fd6500b20a0 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 >> v1:172.21.15.111:6789/0 conn(0x7fd6500b20a0 legacy=0x7fd6500d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 0x7fd63c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd63c003ef0 0x7fd63c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd63c004d00 0x7fd63c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500d39e0 con 0x7fd63c004d00 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500d3cc0 con 0x7fd63c003ef0 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c00b9d0 con 0x7fd6500d1780 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd6552ba640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd63c004d00 0x7fd63c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd654ab9640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 0x7fd63c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.533+0000 7fd64ffff640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd63c003ef0 0x7fd63c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd6552ba640 1 -- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 msgr2=0x7fd63c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd6552ba640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6500d1780 0x7fd63c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd6552ba640 1 -- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd63c003ef0 msgr2=0x7fd63c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd6552ba640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd63c003ef0 0x7fd63c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd6552ba640 1 -- 172.21.15.111:0/3777788488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c003520 con 0x7fd63c004d00 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd6552ba640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd63c004d00 0x7fd63c005180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd640009af0 tx=0x7fd63c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0e8a6acd0c989d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64002c710 con 0x7fd63c004d00 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64002c870 con 0x7fd63c004d00 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.534+0000 7fd64f7fe640 1 -- 172.21.15.111:0/3777788488 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64002cc00 con 0x7fd63c004d00 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd656d1d640 1 -- 172.21.15.111:0/3777788488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd63c004d00 msgr2=0x7fd63c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd656d1d640 1 --2- 172.21.15.111:0/3777788488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd63c004d00 0x7fd63c005180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd640009af0 tx=0x7fd63c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd6552ba640 1 -- 172.21.15.111:0/3777788488 reap_dead start 2023-12-04T05:40:11.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd656d1d640 1 -- 172.21.15.111:0/3777788488 shutdown_connections 2023-12-04T05:40:11.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd656d1d640 1 -- 172.21.15.111:0/3777788488 >> 172.21.15.111:0/3777788488 conn(0x7fd65009c4e0 msgr2=0x7fd6500d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:11.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd656d1d640 1 -- 172.21.15.111:0/3777788488 shutdown_connections 2023-12-04T05:40:11.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.535+0000 7fd656d1d640 1 -- 172.21.15.111:0/3777788488 wait complete. 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.536+0000 7fd656d1d640 1 Processor -- start 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.536+0000 7fd656d1d640 1 -- start start 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd656d1d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd65015c8d0 0x7fd6501f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd656d1d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 0x7fd6500b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd656d1d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd65015da80 0x7fd65015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd656d1d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500b8d50 con 0x7fd6500b8390 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd656d1d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65015ccb0 con 0x7fd65015c8d0 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd656d1d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65015f480 con 0x7fd65015da80 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 0x7fd6500b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 0x7fd6500b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42018/0 (socket says 172.21.15.111:42018) 2023-12-04T05:40:11.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 -- 172.21.15.111:0/2991684072 learned_addr learned my addr 172.21.15.111:0/2991684072 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd6552ba640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd65015da80 0x7fd65015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd654ab9640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd65015c8d0 0x7fd6501f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 -- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd65015da80 msgr2=0x7fd65015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd65015da80 0x7fd65015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 -- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd65015c8d0 msgr2=0x7fd6501f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd65015c8d0 0x7fd6501f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.537+0000 7fd64ffff640 1 -- 172.21.15.111:0/2991684072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65015f720 con 0x7fd6500b8390 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.538+0000 7fd64ffff640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 0x7fd6500b8810 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd6380072c0 tx=0x7fd638007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.538+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800f920 con 0x7fd6500b8390 2023-12-04T05:40:11.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.538+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63800faa0 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.538+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65015fa00 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.538+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800fe50 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.538+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65015ffc0 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.539+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6500693d0 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.539+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd63801d460 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.539+0000 7fd64d7fa640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd63006b2a0 0x7fd63006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.540+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd638089d30 con 0x7fd6500b8390 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.540+0000 7fd654ab9640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd63006b2a0 0x7fd63006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.540+0000 7fd654ab9640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd63006b2a0 0x7fd63006d760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd648014510 tx=0x7fd64800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:11.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.544+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd638054c50 con 0x7fd6500b8390 2023-12-04T05:40:11.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.711+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6500c6a10 con 0x7fd6500b8390 2023-12-04T05:40:11.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.712+0000 7fd64d7fa640 1 -- 172.21.15.111:0/2991684072 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16558 (secure 0 0 0) 0x7fd638058270 con 0x7fd6500b8390 2023-12-04T05:40:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:11.714 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":32,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:10.723653+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd63006b2a0 msgr2=0x7fd63006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd63006b2a0 0x7fd63006d760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd648014510 tx=0x7fd64800b040 comp rx=0 tx=0).stop 2023-12-04T05:40:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 msgr2=0x7fd6500b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 0x7fd6500b8810 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd6380072c0 tx=0x7fd638007780 comp rx=0 tx=0).stop 2023-12-04T05:40:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 shutdown_connections 2023-12-04T05:40:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd63006b2a0 0x7fd63006d760 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd65015da80 0x7fd65015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6500b8390 0x7fd6500b8810 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 --2- 172.21.15.111:0/2991684072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd65015c8d0 0x7fd6501f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 >> 172.21.15.111:0/2991684072 conn(0x7fd65009c4e0 msgr2=0x7fd65006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 shutdown_connections 2023-12-04T05:40:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.714+0000 7fd656d1d640 1 -- 172.21.15.111:0/2991684072 wait complete. 2023-12-04T05:40:11.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:11.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.825+0000 7f3650f0f640 1 Processor -- start 2023-12-04T05:40:11.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.825+0000 7f3650f0f640 1 -- start start 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.825+0000 7f3650f0f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f364c13cee0 0x7f364c06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.825+0000 7f3650f0f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15aba0 0x7f364c06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.825+0000 7f3650f0f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c0aea20 con 0x7f364c140200 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f3650f0f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c13d2c0 con 0x7f364c13cee0 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f3650f0f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c15b3a0 con 0x7f364c15aba0 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f3641d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f364c140200 0x7f364c15a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48762/0 (socket says 172.21.15.111:48762) 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f3641d9b640 1 -- 172.21.15.111:0/3611179177 learned_addr learned my addr 172.21.15.111:0/3611179177 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f364a59c640 1 --2- 172.21.15.111:0/3611179177 >> v2:172.21.15.192:3300/0 conn(0x7f364c13cee0 0x7f364c06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f364ad9d640 1 --2- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15aba0 0x7f364c06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f3649d9b640 1 -- 172.21.15.111:0/3611179177 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938614133 0 0) 0x7f364c0aea20 con 0x7f364c140200 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f3649d9b640 1 -- 172.21.15.111:0/3611179177 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3638003520 con 0x7f364c140200 2023-12-04T05:40:11.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f364a59c640 1 -- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15aba0 msgr2=0x7f364c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f364a59c640 1 --2- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15aba0 0x7f364c06c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f364a59c640 1 -- 172.21.15.111:0/3611179177 >> v1:172.21.15.111:6789/0 conn(0x7f364c140200 legacy=0x7f364c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.826+0000 7f364a59c640 1 -- 172.21.15.111:0/3611179177 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c15b0c0 con 0x7f364c13cee0 2023-12-04T05:40:11.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.827+0000 7f364a59c640 1 --2- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c06c250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f363c0097a0 tx=0x7f363c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2cdd89e45814a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:11.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.827+0000 7f3649d9b640 1 -- 172.21.15.111:0/3611179177 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c02e070 con 0x7f364c13cee0 2023-12-04T05:40:11.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.827+0000 7f3649d9b640 1 -- 172.21.15.111:0/3611179177 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c0315a0 con 0x7f364c13cee0 2023-12-04T05:40:11.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.827+0000 7f3649d9b640 1 -- 172.21.15.111:0/3611179177 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c031910 con 0x7f364c13cee0 2023-12-04T05:40:11.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 -- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 msgr2=0x7f364c06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 --2- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c06c250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f363c0097a0 tx=0x7f363c02b840 comp rx=0 tx=0).stop 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 -- 172.21.15.111:0/3611179177 shutdown_connections 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 --2- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c15aba0 0x7f364c06c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 --2- 172.21.15.111:0/3611179177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c06c250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 -- 172.21.15.111:0/3611179177 >> 172.21.15.111:0/3611179177 conn(0x7f364c06a4c0 msgr2=0x7f364c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 -- 172.21.15.111:0/3611179177 shutdown_connections 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.828+0000 7f3650f0f640 1 -- 172.21.15.111:0/3611179177 wait complete. 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.829+0000 7f3650f0f640 1 Processor -- start 2023-12-04T05:40:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.829+0000 7f3650f0f640 1 -- start start 2023-12-04T05:40:11.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3650f0f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3650f0f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c140200 0x7f364c13eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3650f0f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 0x7f364c13d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3650f0f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c151d10 con 0x7f364c15aba0 2023-12-04T05:40:11.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3650f0f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c138e70 con 0x7f364c13cee0 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3650f0f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c153c20 con 0x7f364c140200 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 0x7f364c13d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 0x7f364c13d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42026/0 (socket says 172.21.15.111:42026) 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 -- 172.21.15.111:0/540828015 learned_addr learned my addr 172.21.15.111:0/540828015 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364a59c640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3641d9b640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c140200 0x7f364c13eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 -- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c140200 msgr2=0x7f364c13eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c140200 0x7f364c13eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 -- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 msgr2=0x7f364c138930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c138930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 -- 172.21.15.111:0/540828015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009400 con 0x7f364c15aba0 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f3641d9b640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c140200 0x7f364c13eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:11.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364a59c640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.830+0000 7f364ad9d640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 0x7f364c13d460 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3644009cb0 tx=0x7f364400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.831+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644009b20 con 0x7f364c15aba0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.831+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364400f040 con 0x7f364c15aba0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.831+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c13f400 con 0x7f364c15aba0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.831+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440146c0 con 0x7f364c15aba0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.831+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c151a30 con 0x7f364c15aba0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.832+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c0693d0 con 0x7f364c15aba0 2023-12-04T05:40:11.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.832+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3644014820 con 0x7f364c15aba0 2023-12-04T05:40:11.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.832+0000 7f36437fe640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f362c06b2a0 0x7f362c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:11.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.833+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f364408aa10 con 0x7f364c15aba0 2023-12-04T05:40:11.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.833+0000 7f364a59c640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f362c06b2a0 0x7f362c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:11.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.833+0000 7f364a59c640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f362c06b2a0 0x7f362c06d760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f363c036000 tx=0x7f363c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:11.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:11.837+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3644055800 con 0x7f364c15aba0 2023-12-04T05:40:12.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.002+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f364c06d170 con 0x7f364c15aba0 2023-12-04T05:40:12.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.003+0000 7f36437fe640 1 -- 172.21.15.111:0/540828015 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16558 (secure 0 0 0) 0x7f3644018090 con 0x7f364c15aba0 2023-12-04T05:40:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:12.005 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":32,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:10.723653+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f362c06b2a0 msgr2=0x7f362c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f362c06b2a0 0x7f362c06d760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f363c036000 tx=0x7f363c03d040 comp rx=0 tx=0).stop 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 msgr2=0x7f364c13d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 0x7f364c13d460 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3644009cb0 tx=0x7f364400c490 comp rx=0 tx=0).stop 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 shutdown_connections 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f362c06b2a0 0x7f362c06d760 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f364c15aba0 0x7f364c13d460 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f364c140200 0x7f364c13eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 --2- 172.21.15.111:0/540828015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f364c13cee0 0x7f364c138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 >> 172.21.15.111:0/540828015 conn(0x7f364c06a4c0 msgr2=0x7f364c0711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 shutdown_connections 2023-12-04T05:40:12.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.005+0000 7f3650f0f640 1 -- 172.21.15.111:0/540828015 wait complete. 2023-12-04T05:40:12.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-04T05:40:12.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.119+0000 7ff150397640 1 Processor -- start 2023-12-04T05:40:12.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.119+0000 7ff150397640 1 -- start start 2023-12-04T05:40:12.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff150397640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff14815a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff150397640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14815b4d0 con 0x7ff14815aca0 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff150397640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14815dc30 con 0x7ff14813cee0 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff150397640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14815e5d0 con 0x7ff14806c940 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14e934640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff14815aca0 0x7ff1480cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48766/0 (socket says 172.21.15.111:48766) 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14e934640 1 -- 172.21.15.111:0/3381229846 learned_addr learned my addr 172.21.15.111:0/3381229846 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d932640 1 --2- 172.21.15.111:0/3381229846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff14815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d131640 1 -- 172.21.15.111:0/3381229846 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469019873 0 0) 0x7ff14815b4d0 con 0x7ff14815aca0 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d131640 1 -- 172.21.15.111:0/3381229846 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff134003520 con 0x7ff14815aca0 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d131640 1 -- 172.21.15.111:0/3381229846 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2491359132 0 0) 0x7ff134003520 con 0x7ff14815aca0 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d932640 1 -- 172.21.15.111:0/3381229846 >> v1:172.21.15.192:6789/0 conn(0x7ff14813cee0 legacy=0x7ff14806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d932640 1 -- 172.21.15.111:0/3381229846 >> v1:172.21.15.111:6789/0 conn(0x7ff14815aca0 legacy=0x7ff1480cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.120+0000 7ff14d932640 1 -- 172.21.15.111:0/3381229846 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14815df10 con 0x7ff14806c940 2023-12-04T05:40:12.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.121+0000 7ff14d932640 1 --2- 172.21.15.111:0/3381229846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff14815a760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1380097d0 tx=0x7ff13802b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7a6131651d2f2b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.121+0000 7ff14d131640 1 -- 172.21.15.111:0/3381229846 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1400019f0 con 0x7ff14806c940 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.121+0000 7ff14d131640 1 -- 172.21.15.111:0/3381229846 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13802e050 con 0x7ff14806c940 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.121+0000 7ff14d131640 1 -- 172.21.15.111:0/3381229846 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138030710 con 0x7ff14806c940 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 -- 172.21.15.111:0/3381229846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 msgr2=0x7ff14815a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 --2- 172.21.15.111:0/3381229846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff14815a760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1380097d0 tx=0x7ff13802b5d0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 -- 172.21.15.111:0/3381229846 shutdown_connections 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 --2- 172.21.15.111:0/3381229846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff14815a760 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 -- 172.21.15.111:0/3381229846 >> 172.21.15.111:0/3381229846 conn(0x7ff14806a4c0 msgr2=0x7ff14806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 -- 172.21.15.111:0/3381229846 shutdown_connections 2023-12-04T05:40:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.122+0000 7ff150397640 1 -- 172.21.15.111:0/3381229846 wait complete. 2023-12-04T05:40:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.123+0000 7ff150397640 1 Processor -- start 2023-12-04T05:40:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.123+0000 7ff150397640 1 -- start start 2023-12-04T05:40:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff150397640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff1481f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff150397640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 0x7ff1481f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff150397640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff14815aca0 0x7ff1480703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff150397640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1481f7260 con 0x7ff14813cee0 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff150397640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148137950 con 0x7ff14815aca0 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff150397640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff148137c30 con 0x7ff14806c940 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 0x7ff1481f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 0x7ff1481f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42042/0 (socket says 172.21.15.111:42042) 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 -- 172.21.15.111:0/338121309 learned_addr learned my addr 172.21.15.111:0/338121309 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14e133640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff1481f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14e934640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff14815aca0 0x7ff1480703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 -- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 msgr2=0x7ff1481f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff1481f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 -- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff14815aca0 msgr2=0x7ff1480703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff14815aca0 0x7ff1480703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 -- 172.21.15.111:0/338121309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138009400 con 0x7ff14813cee0 2023-12-04T05:40:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff14d932640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 0x7ff1481f6d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff13802b610 tx=0x7ff1380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.124+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13803d890 con 0x7ff14813cee0 2023-12-04T05:40:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.125+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13803d9f0 con 0x7ff14813cee0 2023-12-04T05:40:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.125+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13803dda0 con 0x7ff14813cee0 2023-12-04T05:40:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.125+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff148137f10 con 0x7ff14813cee0 2023-12-04T05:40:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.125+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1481384d0 con 0x7ff14813cee0 2023-12-04T05:40:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.125+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14813a340 con 0x7ff14813cee0 2023-12-04T05:40:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.126+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff138039040 con 0x7ff14813cee0 2023-12-04T05:40:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.126+0000 7ff13effd640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff12c06b2a0 0x7ff12c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.126+0000 7ff14e133640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff12c06b2a0 0x7ff12c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.126+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1380abba0 con 0x7ff14813cee0 2023-12-04T05:40:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.127+0000 7ff14e133640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff12c06b2a0 0x7ff12c06d760 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff144000f60 tx=0x7ff144009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.130+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff138076aa0 con 0x7ff14813cee0 2023-12-04T05:40:12.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:12.337+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff1481386c0 con 0x7ff14813cee0 2023-12-04T05:40:13.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.202+0000 7ff13effd640 1 -- 172.21.15.111:0/338121309 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff13804d0b0 con 0x7ff14813cee0 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff12c06b2a0 msgr2=0x7ff12c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff12c06b2a0 0x7ff12c06d760 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff144000f60 tx=0x7ff144009040 comp rx=0 tx=0).stop 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 msgr2=0x7ff1481f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 0x7ff1481f6d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff13802b610 tx=0x7ff1380046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 shutdown_connections 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff12c06b2a0 0x7ff12c06d760 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff14815aca0 0x7ff1480703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff14813cee0 0x7ff1481f6d20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 --2- 172.21.15.111:0/338121309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff14806c940 0x7ff1481f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 >> 172.21.15.111:0/338121309 conn(0x7ff14806a4c0 msgr2=0x7ff14813b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:13.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 shutdown_connections 2023-12-04T05:40:13.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.205+0000 7ff150397640 1 -- 172.21.15.111:0/338121309 wait complete. 2023-12-04T05:40:13.220 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-04T05:40:13.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-04T05:40:13.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.327+0000 7fa158a90640 1 Processor -- start 2023-12-04T05:40:13.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.327+0000 7fa158a90640 1 -- start start 2023-12-04T05:40:13.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa158a90640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa154168380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa158a90640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15415da00 con 0x7fa1540d18a0 2023-12-04T05:40:13.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa158a90640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa154161d60 con 0x7fa154146a60 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa158a90640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154161b70 con 0x7fa154161700 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa151d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa1540d18a0 0x7fa154160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48770/0 (socket says 172.21.15.111:48770) 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa151d9b640 1 -- 172.21.15.111:0/3387283176 learned_addr learned my addr 172.21.15.111:0/3387283176 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa152d9d640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa154168380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062631402 0 0) 0x7fa15415da00 con 0x7fa1540d18a0 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.328+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa13c003520 con 0x7fa1540d18a0 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268611527 0 0) 0x7fa13c003520 con 0x7fa1540d18a0 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 msgr2=0x7fa154168380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa154168380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 >> v1:172.21.15.192:6789/0 conn(0x7fa154146a60 legacy=0x7fa1540d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15415dcc0 con 0x7fa1540d18a0 2023-12-04T05:40:13.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fa148001670 con 0x7fa1540d18a0 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 >> v1:172.21.15.111:6789/0 conn(0x7fa1540d18a0 legacy=0x7fa154160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa13c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa13c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa13c004d00 0x7fa13c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00ba30 con 0x7fa13c0039a0 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00bb70 con 0x7fa13c004d00 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c00bd30 con 0x7fa154161700 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15259c640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa13c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa151d9b640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa13c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa152d9d640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa13c004d00 0x7fa13c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15259c640 1 -- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 msgr2=0x7fa13c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15259c640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa13c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15259c640 1 -- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa13c004d00 msgr2=0x7fa13c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15259c640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa13c004d00 0x7fa13c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa15259c640 1 -- 172.21.15.111:0/3387283176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c003520 con 0x7fa13c0039a0 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa151d9b640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa154161700 0x7fa13c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.329+0000 7fa152d9d640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa13c004d00 0x7fa13c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.330+0000 7fa15259c640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa13c003d80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa144001f60 tx=0x7fa13c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cc93fa90dc4ff9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.330+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14402c710 con 0x7fa13c0039a0 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.330+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14402c890 con 0x7fa13c0039a0 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.330+0000 7fa15159a640 1 -- 172.21.15.111:0/3387283176 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14402cc40 con 0x7fa13c0039a0 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa158a90640 1 -- 172.21.15.111:0/3387283176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 msgr2=0x7fa13c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa158a90640 1 --2- 172.21.15.111:0/3387283176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa13c003d80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa144001f60 tx=0x7fa13c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa152d9d640 1 -- 172.21.15.111:0/3387283176 reap_dead start 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa158a90640 1 -- 172.21.15.111:0/3387283176 shutdown_connections 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa158a90640 1 -- 172.21.15.111:0/3387283176 >> 172.21.15.111:0/3387283176 conn(0x7fa15406a6a0 msgr2=0x7fa15406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:13.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa158a90640 1 -- 172.21.15.111:0/3387283176 shutdown_connections 2023-12-04T05:40:13.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.331+0000 7fa158a90640 1 -- 172.21.15.111:0/3387283176 wait complete. 2023-12-04T05:40:13.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.332+0000 7fa158a90640 1 Processor -- start 2023-12-04T05:40:13.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.332+0000 7fa158a90640 1 -- start start 2023-12-04T05:40:13.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa1540d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa15415c4e0 0x7fa15415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540cb4a0 0x7fa1540cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540cce90 con 0x7fa13c0039a0 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540cc210 con 0x7fa15415c4e0 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1540cc4f0 con 0x7fa1540cb4a0 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa1540d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa1540d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42058/0 (socket says 172.21.15.111:42058) 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 -- 172.21.15.111:0/2696459718 learned_addr learned my addr 172.21.15.111:0/2696459718 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 -- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540cb4a0 msgr2=0x7fa1540cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540cb4a0 0x7fa1540cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa151d9b640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa15415c4e0 0x7fa15415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 -- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa15415c4e0 msgr2=0x7fa15415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa15415c4e0 0x7fa15415c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 -- 172.21.15.111:0/2696459718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540cc7d0 con 0x7fa13c0039a0 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa151d9b640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa15415c4e0 0x7fa15415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:13.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa15259c640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa1540d05d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa144002080 tx=0x7fa1440015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144003d80 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14402ce80 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1540ccab0 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.333+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144005270 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.334+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1540cbe60 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.335+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15406d000 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.335+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa14403b050 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.335+0000 7fa142ffd640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa13406b2a0 0x7fa13406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.335+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1440ab6c0 con 0x7fa13c0039a0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.335+0000 7fa151d9b640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa13406b2a0 0x7fa13406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:13.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.336+0000 7fa151d9b640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa13406b2a0 0x7fa13406d760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa148001eb0 tx=0x7fa148001de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:13.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.339+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa144076670 con 0x7fa13c0039a0 2023-12-04T05:40:13.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:13.541+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa1540693d0 con 0x7fa13c0039a0 2023-12-04T05:40:14.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.411+0000 7fa142ffd640 1 -- 172.21.15.111:0/2696459718 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa144079c90 con 0x7fa13c0039a0 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa13406b2a0 msgr2=0x7fa13406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa13406b2a0 0x7fa13406d760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa148001eb0 tx=0x7fa148001de0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 msgr2=0x7fa1540d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa1540d05d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa144002080 tx=0x7fa1440015b0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 shutdown_connections 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa13406b2a0 0x7fa13406d760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1540cb4a0 0x7fa1540cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa15415c4e0 0x7fa15415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 --2- 172.21.15.111:0/2696459718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa13c0039a0 0x7fa1540d05d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 >> 172.21.15.111:0/2696459718 conn(0x7fa15406a6a0 msgr2=0x7fa1540b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 shutdown_connections 2023-12-04T05:40:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.413+0000 7fa158a90640 1 -- 172.21.15.111:0/2696459718 wait complete. 2023-12-04T05:40:14.426 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-04T05:40:14.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.527+0000 7f27015ec640 1 Processor -- start 2023-12-04T05:40:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.527+0000 7f27015ec640 1 -- start start 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f27015ec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26fc0b20a0 0x7f26fc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f27015ec640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f26fc0b2ad0 0x7f26fc0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f27015ec640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc0cdcb0 con 0x7f26fc0d1780 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f27015ec640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc0d39e0 con 0x7f26fc0b2ad0 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f27015ec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc0d3cc0 con 0x7f26fc0b20a0 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f26faffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f26fc0d1780 0x7f26fc0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48782/0 (socket says 172.21.15.111:48782) 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f26faffd640 1 -- 172.21.15.111:0/3671192454 learned_addr learned my addr 172.21.15.111:0/3671192454 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.528+0000 7f26fb7fe640 1 --2- 172.21.15.111:0/3671192454 >> v2:172.21.15.111:3300/0 conn(0x7f26fc0b2ad0 0x7f26fc0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26fa7fc640 1 --2- 172.21.15.111:0/3671192454 >> v2:172.21.15.192:3300/0 conn(0x7f26fc0b20a0 0x7f26fc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26f9ffb640 1 -- 172.21.15.111:0/3671192454 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512552932 0 0) 0x7f26fc0cdcb0 con 0x7f26fc0d1780 2023-12-04T05:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26f9ffb640 1 -- 172.21.15.111:0/3671192454 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e4003520 con 0x7f26fc0d1780 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26fb7fe640 1 -- 172.21.15.111:0/3671192454 >> v2:172.21.15.192:3300/0 conn(0x7f26fc0b20a0 msgr2=0x7f26fc0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26fb7fe640 1 --2- 172.21.15.111:0/3671192454 >> v2:172.21.15.192:3300/0 conn(0x7f26fc0b20a0 0x7f26fc0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26fb7fe640 1 -- 172.21.15.111:0/3671192454 >> v1:172.21.15.111:6789/0 conn(0x7f26fc0d1780 legacy=0x7f26fc0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26fb7fe640 1 -- 172.21.15.111:0/3671192454 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0cdf50 con 0x7f26fc0b2ad0 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.529+0000 7f26fb7fe640 1 --2- 172.21.15.111:0/3671192454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0b2ad0 0x7f26fc0b2f00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26fc162190 tx=0x7f26f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44c2daa996638ea server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f26f9ffb640 1 -- 172.21.15.111:0/3671192454 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f402e070 con 0x7f26fc0b2ad0 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f26f9ffb640 1 -- 172.21.15.111:0/3671192454 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40315a0 con 0x7f26fc0b2ad0 2023-12-04T05:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f26f9ffb640 1 -- 172.21.15.111:0/3671192454 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4031930 con 0x7f26fc0b2ad0 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 -- 172.21.15.111:0/3671192454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0b2ad0 msgr2=0x7f26fc0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 --2- 172.21.15.111:0/3671192454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0b2ad0 0x7f26fc0b2f00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26fc162190 tx=0x7f26f402b840 comp rx=0 tx=0).stop 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 -- 172.21.15.111:0/3671192454 shutdown_connections 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 --2- 172.21.15.111:0/3671192454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0b2ad0 0x7f26fc0b2f00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 --2- 172.21.15.111:0/3671192454 >> v2:172.21.15.192:3300/0 conn(0x7f26fc0b20a0 0x7f26fc0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 -- 172.21.15.111:0/3671192454 >> 172.21.15.111:0/3671192454 conn(0x7f26fc09c4e0 msgr2=0x7f26fc0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 -- 172.21.15.111:0/3671192454 shutdown_connections 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.530+0000 7f27015ec640 1 -- 172.21.15.111:0/3671192454 wait complete. 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.531+0000 7f27015ec640 1 Processor -- start 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.531+0000 7f27015ec640 1 -- start start 2023-12-04T05:40:14.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f27015ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26fc0b20a0 0x7f26fc1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f27015ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26fc0b2ad0 0x7f26fc156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f27015ec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 0x7f26fc0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f27015ec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b87f0 con 0x7f26fc0d1780 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f27015ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b5dc0 con 0x7f26fc0b20a0 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f27015ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b6940 con 0x7f26fc0b2ad0 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 0x7f26fc0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 0x7f26fc0cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42072/0 (socket says 172.21.15.111:42072) 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 -- 172.21.15.111:0/2098909359 learned_addr learned my addr 172.21.15.111:0/2098909359 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26faffd640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26fc0b20a0 0x7f26fc1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fa7fc640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26fc0b2ad0 0x7f26fc156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 -- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26fc0b2ad0 msgr2=0x7f26fc156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26fc0b2ad0 0x7f26fc156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 -- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26fc0b20a0 msgr2=0x7f26fc1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26fc0b20a0 0x7f26fc1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.532+0000 7f26fb7fe640 1 -- 172.21.15.111:0/2098909359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4009400 con 0x7f26fc0d1780 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.533+0000 7f26fb7fe640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 0x7f26fc0cc2e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f26f402be50 tx=0x7f26f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.533+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4032840 con 0x7f26fc0d1780 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.533+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40329a0 con 0x7f26fc0d1780 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.533+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4032d30 con 0x7f26fc0d1780 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.533+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc158de0 con 0x7f26fc0d1780 2023-12-04T05:40:14.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.533+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc0b62f0 con 0x7f26fc0d1780 2023-12-04T05:40:14.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.534+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc0693d0 con 0x7f26fc0d1780 2023-12-04T05:40:14.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.534+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f26f404e430 con 0x7f26fc0d1780 2023-12-04T05:40:14.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.535+0000 7f26e37fe640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26e806b2a0 0x7f26e806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.535+0000 7f26faffd640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26e806b2a0 0x7f26e806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.535+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26f40abb00 con 0x7f26fc0d1780 2023-12-04T05:40:14.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.535+0000 7f26faffd640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26e806b2a0 0x7f26e806d760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26ec001560 tx=0x7f26ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:14.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.539+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26f4076970 con 0x7f26fc0d1780 2023-12-04T05:40:14.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.742+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26fc0b6a80 con 0x7f26fc0d1780 2023-12-04T05:40:14.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.742+0000 7f26e37fe640 1 -- 172.21.15.111:0/2098909359 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f26f4079f90 con 0x7f26fc0d1780 2023-12-04T05:40:14.744 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:14.744 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:14.744 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26e806b2a0 msgr2=0x7f26e806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26e806b2a0 0x7f26e806d760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f26ec001560 tx=0x7f26ec009040 comp rx=0 tx=0).stop 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 msgr2=0x7f26fc0cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 0x7f26fc0cc2e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f26f402be50 tx=0x7f26f40046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 shutdown_connections 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f26e806b2a0 0x7f26e806d760 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26fc0d1780 0x7f26fc0cc2e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26fc0b2ad0 0x7f26fc156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 --2- 172.21.15.111:0/2098909359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26fc0b20a0 0x7f26fc1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 >> 172.21.15.111:0/2098909359 conn(0x7f26fc09c4e0 msgr2=0x7f26fc0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:14.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 shutdown_connections 2023-12-04T05:40:14.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.744+0000 7f27015ec640 1 -- 172.21.15.111:0/2098909359 wait complete. 2023-12-04T05:40:14.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:14.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.854+0000 7f57d68da640 1 Processor -- start 2023-12-04T05:40:14.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.854+0000 7f57d68da640 1 -- start start 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57d68da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 0x7f57d0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57d68da640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d015fc10 con 0x7f57d00721b0 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57d68da640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d0070bd0 con 0x7f57d0144430 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57d68da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0144810 con 0x7f57d015f7b0 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57d4e77640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f57d00721b0 0x7f57d006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48790/0 (socket says 172.21.15.111:48790) 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57d4e77640 1 -- 172.21.15.111:0/1002672830 learned_addr learned my addr 172.21.15.111:0/1002672830 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753780209 0 0) 0x7f57d015fc10 con 0x7f57d00721b0 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b8003520 con 0x7f57d00721b0 2023-12-04T05:40:14.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.855+0000 7f57cf7fe640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 0x7f57d0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234864477 0 0) 0x7f57b8003520 con 0x7f57d00721b0 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 msgr2=0x7f57d0071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 0x7f57d0071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 >> v1:172.21.15.192:6789/0 conn(0x7f57d0144430 legacy=0x7f57d00703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0070eb0 con 0x7f57d00721b0 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f57c80019f0 con 0x7f57d00721b0 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 >> v1:172.21.15.111:6789/0 conn(0x7f57d00721b0 legacy=0x7f57d006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57b80047c0 0x7f57b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 0x7f57b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0144810 con 0x7f57b80039a0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b800ba30 con 0x7f57b80047c0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b800bbf0 con 0x7f57d015f7b0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cffff640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cf7fe640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57b80047c0 0x7f57b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57d4e77640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 0x7f57b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cffff640 1 -- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 msgr2=0x7f57b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cffff640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d015f7b0 0x7f57b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cffff640 1 -- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57b80047c0 msgr2=0x7f57b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cffff640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57b80047c0 0x7f57b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.856+0000 7f57cffff640 1 -- 172.21.15.111:0/1002672830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8003520 con 0x7f57b80039a0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57cffff640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57b8003dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f57c0001720 tx=0x7f57b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6551bbc09cb57751 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00052d0 con 0x7f57b80039a0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c002e050 con 0x7f57b80039a0 2023-12-04T05:40:14.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57ceffd640 1 -- 172.21.15.111:0/1002672830 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00035f0 con 0x7f57b80039a0 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d68da640 1 -- 172.21.15.111:0/1002672830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 msgr2=0x7f57b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d68da640 1 --2- 172.21.15.111:0/1002672830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57b8003dc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f57c0001720 tx=0x7f57b80057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d4e77640 1 -- 172.21.15.111:0/1002672830 reap_dead start 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d68da640 1 -- 172.21.15.111:0/1002672830 shutdown_connections 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d68da640 1 -- 172.21.15.111:0/1002672830 >> 172.21.15.111:0/1002672830 conn(0x7f57d006a4c0 msgr2=0x7f57d006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d68da640 1 -- 172.21.15.111:0/1002672830 shutdown_connections 2023-12-04T05:40:14.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.857+0000 7f57d68da640 1 -- 172.21.15.111:0/1002672830 wait complete. 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 Processor -- start 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 -- start start 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57d015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d00c9130 0x7f57d00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d00b39c0 0x7f57d00c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00c8d10 con 0x7f57b80039a0 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d015afb0 con 0x7f57d00b39c0 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.859+0000 7f57d68da640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00b3ff0 con 0x7f57d00c9130 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57d015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57d015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42092/0 (socket says 172.21.15.111:42092) 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 -- 172.21.15.111:0/2555250578 learned_addr learned my addr 172.21.15.111:0/2555250578 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cf7fe640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d00c9130 0x7f57d00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57d4e77640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d00b39c0 0x7f57d00c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 -- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d00c9130 msgr2=0x7f57d00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d00c9130 0x7f57d00b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 -- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d00b39c0 msgr2=0x7f57d00c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d00b39c0 0x7f57d00c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 -- 172.21.15.111:0/2555250578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d00c8370 con 0x7f57b80039a0 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cf7fe640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d00c9130 0x7f57d00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57d4e77640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d00b39c0 0x7f57d00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57cffff640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57d015aa70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f57c0008fd0 tx=0x7f57c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00076c0 con 0x7f57b80039a0 2023-12-04T05:40:14.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c0007a30 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0006be0 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d00c8650 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.860+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d013ccc0 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.861+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d00c8f80 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.862+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f57c0006d40 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.862+0000 7f57ccff9640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f57bc06b2d0 0x7f57bc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.862+0000 7f57cf7fe640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f57bc06b2d0 0x7f57bc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.862+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57c00ab940 con 0x7f57b80039a0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.862+0000 7f57cf7fe640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f57bc06b2d0 0x7f57bc06d790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57d00cd4a0 tx=0x7f57c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:14.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:14.866+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57c007a240 con 0x7f57b80039a0 2023-12-04T05:40:15.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.033+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d00b3da0 con 0x7f57b80039a0 2023-12-04T05:40:15.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.034+0000 7f57ccff9640 1 -- 172.21.15.111:0/2555250578 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f57c004b090 con 0x7f57b80039a0 2023-12-04T05:40:15.035 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:15.035 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:15.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f57bc06b2d0 msgr2=0x7f57bc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f57bc06b2d0 0x7f57bc06d790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57d00cd4a0 tx=0x7f57c4008040 comp rx=0 tx=0).stop 2023-12-04T05:40:15.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 msgr2=0x7f57d015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57d015aa70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f57c0008fd0 tx=0x7f57c0001120 comp rx=0 tx=0).stop 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 shutdown_connections 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f57bc06b2d0 0x7f57bc06d790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d00b39c0 0x7f57d00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d00c9130 0x7f57d00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 --2- 172.21.15.111:0/2555250578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b80039a0 0x7f57d015aa70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 >> 172.21.15.111:0/2555250578 conn(0x7f57d006a4c0 msgr2=0x7f57d015b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 shutdown_connections 2023-12-04T05:40:15.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.036+0000 7f57d68da640 1 -- 172.21.15.111:0/2555250578 wait complete. 2023-12-04T05:40:15.048 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-04T05:40:15.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:40:15.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.146+0000 7f473e63d640 1 Processor -- start 2023-12-04T05:40:15.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.146+0000 7f473e63d640 1 -- start start 2023-12-04T05:40:15.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.147+0000 7f473e63d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f473806e7e0 0x7f473806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.147+0000 7f473e63d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380cfaf0 0x7f47380cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.147+0000 7f473e63d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473806f100 con 0x7f473815ed10 2023-12-04T05:40:15.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.147+0000 7f473e63d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473815fc30 con 0x7f473806e7e0 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.147+0000 7f473e63d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473813e380 con 0x7f47380cfaf0 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f4737fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f473806e7e0 0x7f473806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f473cbda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380cfaf0 0x7f47380cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f47377fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f473815ed10 0x7f473815f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:49194/0 (socket says 172.21.15.111:49194) 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f47377fe640 1 -- 172.21.15.111:0/3634277283 learned_addr learned my addr 172.21.15.111:0/3634277283 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f4737fff640 1 -- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380cfaf0 msgr2=0x7f47380cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f4737fff640 1 --2- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380cfaf0 0x7f47380cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f4737fff640 1 -- 172.21.15.111:0/3634277283 >> v1:172.21.15.192:6789/0 conn(0x7f473815ed10 legacy=0x7f473815f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f4737fff640 1 -- 172.21.15.111:0/3634277283 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473815b810 con 0x7f473806e7e0 2023-12-04T05:40:15.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.148+0000 7f4736ffd640 1 -- 172.21.15.111:0/3634277283 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219387049 0 0) 0x7f473806f100 con 0x7f473815ed10 2023-12-04T05:40:15.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f4737fff640 1 --2- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f473806e7e0 0x7f473806ebc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47280097a0 tx=0x7f472802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8e98be1962b60c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f4736ffd640 1 -- 172.21.15.111:0/3634277283 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472802e070 con 0x7f473806e7e0 2023-12-04T05:40:15.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f4736ffd640 1 -- 172.21.15.111:0/3634277283 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47280315a0 con 0x7f473806e7e0 2023-12-04T05:40:15.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f4736ffd640 1 -- 172.21.15.111:0/3634277283 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728031930 con 0x7f473806e7e0 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 -- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f473806e7e0 msgr2=0x7f473806ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 --2- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f473806e7e0 0x7f473806ebc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47280097a0 tx=0x7f472802b840 comp rx=0 tx=0).stop 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 -- 172.21.15.111:0/3634277283 shutdown_connections 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 --2- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380cfaf0 0x7f47380cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 --2- 172.21.15.111:0/3634277283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f473806e7e0 0x7f473806ebc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 -- 172.21.15.111:0/3634277283 >> 172.21.15.111:0/3634277283 conn(0x7f473806a8b0 msgr2=0x7f473806acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.149+0000 7f473e63d640 1 -- 172.21.15.111:0/3634277283 shutdown_connections 2023-12-04T05:40:15.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.150+0000 7f473e63d640 1 -- 172.21.15.111:0/3634277283 wait complete. 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.151+0000 7f473e63d640 1 Processor -- start 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.151+0000 7f473e63d640 1 -- start start 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.151+0000 7f473e63d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 0x7f4738157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.151+0000 7f473e63d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380cfaf0 0x7f473813f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.151+0000 7f473e63d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f473815ed10 0x7f473813f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.151+0000 7f473e63d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738153730 con 0x7f473806e7e0 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f473e63d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738153a10 con 0x7f47380cfaf0 2023-12-04T05:40:15.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f473e63d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4738153cf0 con 0x7f473815ed10 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 0x7f4738157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 0x7f4738157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42100/0 (socket says 172.21.15.111:42100) 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 -- 172.21.15.111:0/2839966335 learned_addr learned my addr 172.21.15.111:0/2839966335 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f47377fe640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380cfaf0 0x7f473813f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f473cbda640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f473815ed10 0x7f473813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 -- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f473815ed10 msgr2=0x7f473813f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f473815ed10 0x7f473813f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 -- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380cfaf0 msgr2=0x7f473813f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380cfaf0 0x7f473813f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 -- 172.21.15.111:0/2839966335 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728009400 con 0x7f473806e7e0 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4737fff640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 0x7f4738157430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4728000c00 tx=0x7f47280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728004230 con 0x7f473806e7e0 2023-12-04T05:40:15.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47280043b0 con 0x7f473806e7e0 2023-12-04T05:40:15.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4738153fd0 con 0x7f473806e7e0 2023-12-04T05:40:15.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728004520 con 0x7f473806e7e0 2023-12-04T05:40:15.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.152+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47381533e0 con 0x7f473806e7e0 2023-12-04T05:40:15.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.153+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47380693d0 con 0x7f473806e7e0 2023-12-04T05:40:15.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.154+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4728032880 con 0x7f473806e7e0 2023-12-04T05:40:15.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.154+0000 7f4734ff9640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f472406b2a0 0x7f472406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.154+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47280abc80 con 0x7f473806e7e0 2023-12-04T05:40:15.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.154+0000 7f47377fe640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f472406b2a0 0x7f472406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.154+0000 7f47377fe640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f472406b2a0 0x7f472406d760 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f47380b2be0 tx=0x7f472c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.158+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4728076b00 con 0x7f473806e7e0 2023-12-04T05:40:15.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.323+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47380d0650 con 0x7f472406b2a0 2023-12-04T05:40:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.331+0000 7f4734ff9640 1 -- 172.21.15.111:0/2839966335 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277427 (secure 0 0 0) 0x7f47380d0650 con 0x7f472406b2a0 2023-12-04T05:40:15.337 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:15.338 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f472406b2a0 msgr2=0x7f472406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f472406b2a0 0x7f472406d760 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f47380b2be0 tx=0x7f472c007d50 comp rx=0 tx=0).stop 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 msgr2=0x7f4738157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 0x7f4738157430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4728000c00 tx=0x7f47280046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 shutdown_connections 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f472406b2a0 0x7f472406d760 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f473815ed10 0x7f473813f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380cfaf0 0x7f473813f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 --2- 172.21.15.111:0/2839966335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473806e7e0 0x7f4738157430 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 >> 172.21.15.111:0/2839966335 conn(0x7f473806a8b0 msgr2=0x7f47380cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 shutdown_connections 2023-12-04T05:40:15.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.334+0000 7f473e63d640 1 -- 172.21.15.111:0/2839966335 wait complete. 2023-12-04T05:40:15.346 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-12-04T05:40:14.483579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219904,"kb_used_data":3700,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754816,"statfs":{"total":773094113280,"available":772868931584,"internally_reserved":0,"allocated":3788800,"data_stored":2157953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.104941"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383186+0000","last_change":"2023-12-04T05:40:09.383186+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383186+0000","last_undegraded":"2023-12-04T05:40:09.383186+0000","last_fullsized":"2023-12-04T05:40:09.383186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382869+0000","last_change":"2023-12-04T05:40:09.382869+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382869+0000","last_undegraded":"2023-12-04T05:40:09.382869+0000","last_fullsized":"2023-12-04T05:40:09.382869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381342+0000","last_change":"2023-12-04T05:40:09.381342+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381342+0000","last_undegraded":"2023-12-04T05:40:09.381342+0000","last_fullsized":"2023-12-04T05:40:09.381342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381105+0000","last_change":"2023-12-04T05:40:09.381105+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381105+0000","last_undegraded":"2023-12-04T05:40:09.381105+0000","last_fullsized":"2023-12-04T05:40:09.381105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380752+0000","last_change":"2023-12-04T05:40:09.380752+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380752+0000","last_undegraded":"2023-12-04T05:40:09.380752+0000","last_fullsized":"2023-12-04T05:40:09.380752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383269+0000","last_change":"2023-12-04T05:40:09.383269+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383269+0000","last_undegraded":"2023-12-04T05:40:09.383269+0000","last_fullsized":"2023-12-04T05:40:09.383269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380659+0000","last_change":"2023-12-04T05:40:09.380659+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380659+0000","last_undegraded":"2023-12-04T05:40:09.380659+0000","last_fullsized":"2023-12-04T05:40:09.380659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382723+0000","last_change":"2023-12-04T05:40:09.382723+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382723+0000","last_undegraded":"2023-12-04T05:40:09.382723+0000","last_fullsized":"2023-12-04T05:40:09.382723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381605+0000","last_change":"2023-12-04T05:40:09.381605+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381605+0000","last_undegraded":"2023-12-04T05:40:09.381605+0000","last_fullsized":"2023-12-04T05:40:09.381605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382799+0000","last_change":"2023-12-04T05:40:09.382799+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382799+0000","last_undegraded":"2023-12-04T05:40:09.382799+0000","last_fullsized":"2023-12-04T05:40:09.382799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380588+0000","last_change":"2023-12-04T05:40:09.380588+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380588+0000","last_undegraded":"2023-12-04T05:40:09.380588+0000","last_fullsized":"2023-12-04T05:40:09.380588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384407+0000","last_change":"2023-12-04T05:40:09.384407+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384407+0000","last_undegraded":"2023-12-04T05:40:09.384407+0000","last_fullsized":"2023-12-04T05:40:09.384407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382706+0000","last_change":"2023-12-04T05:40:09.382706+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382706+0000","last_undegraded":"2023-12-04T05:40:09.382706+0000","last_fullsized":"2023-12-04T05:40:09.382706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383049+0000","last_change":"2023-12-04T05:40:09.383049+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383049+0000","last_undegraded":"2023-12-04T05:40:09.383049+0000","last_fullsized":"2023-12-04T05:40:09.383049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384677+0000","last_change":"2023-12-04T05:40:09.384677+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384677+0000","last_undegraded":"2023-12-04T05:40:09.384677+0000","last_fullsized":"2023-12-04T05:40:09.384677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380488+0000","last_change":"2023-12-04T05:40:09.380488+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380488+0000","last_undegraded":"2023-12-04T05:40:09.380488+0000","last_fullsized":"2023-12-04T05:40:09.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380417+0000","last_change":"2023-12-04T05:40:09.380417+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380417+0000","last_undegraded":"2023-12-04T05:40:09.380417+0000","last_fullsized":"2023-12-04T05:40:09.380417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380404+0000","last_change":"2023-12-04T05:40:09.380404+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380404+0000","last_undegraded":"2023-12-04T05:40:09.380404+0000","last_fullsized":"2023-12-04T05:40:09.380404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383218+0000","last_change":"2023-12-04T05:40:09.383218+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383218+0000","last_undegraded":"2023-12-04T05:40:09.383218+0000","last_fullsized":"2023-12-04T05:40:09.383218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381657+0000","last_change":"2023-12-04T05:40:09.381657+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381657+0000","last_undegraded":"2023-12-04T05:40:09.381657+0000","last_fullsized":"2023-12-04T05:40:09.381657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382052+0000","last_change":"2023-12-04T05:40:09.382052+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382052+0000","last_undegraded":"2023-12-04T05:40:09.382052+0000","last_fullsized":"2023-12-04T05:40:09.382052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384013+0000","last_change":"2023-12-04T05:40:09.384013+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384013+0000","last_undegraded":"2023-12-04T05:40:09.384013+0000","last_fullsized":"2023-12-04T05:40:09.384013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383973+0000","last_change":"2023-12-04T05:40:09.383973+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383973+0000","last_undegraded":"2023-12-04T05:40:09.383973+0000","last_fullsized":"2023-12-04T05:40:09.383973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383973+0000","last_change":"2023-12-04T05:40:09.383973+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383973+0000","last_undegraded":"2023-12-04T05:40:09.383973+0000","last_fullsized":"2023-12-04T05:40:09.383973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380341+0000","last_change":"2023-12-04T05:40:09.380341+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380341+0000","last_undegraded":"2023-12-04T05:40:09.380341+0000","last_fullsized":"2023-12-04T05:40:09.380341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382605+0000","last_change":"2023-12-04T05:40:09.382605+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382605+0000","last_undegraded":"2023-12-04T05:40:09.382605+0000","last_fullsized":"2023-12-04T05:40:09.382605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383277+0000","last_change":"2023-12-04T05:40:09.383277+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383277+0000","last_undegraded":"2023-12-04T05:40:09.383277+0000","last_fullsized":"2023-12-04T05:40:09.383277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-04T05:40:05.524663+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:40:05.524663+0000","last_peered":"2023-12-04T05:40:05.524663+0000","last_clean":"2023-12-04T05:40:05.524663+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:40:05.524663+0000","last_undegraded":"2023-12-04T05:40:05.524663+0000","last_fullsized":"2023-12-04T05:40:05.524663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":711793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.435}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.5}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.629}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.501}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46899999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46400000000000002}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54600000000000004}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.496}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.751}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.315},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26500000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.751}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.253},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.27200000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.307},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.318},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.371}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.39100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.34200000000000003}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.375}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.314},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:40:15.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:40:15.367 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:40:15.366+0000 7f32d6815640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-04T05:40:15.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.466+0000 7f6965f41640 1 Processor -- start 2023-12-04T05:40:15.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f6965f41640 1 -- start start 2023-12-04T05:40:15.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f6965f41640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69600d1780 0x7f69600d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f6965f41640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f69600b20a0 0x7f69600d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f6965f41640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69600cdcb0 con 0x7f69600b2ad0 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f6965f41640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69600d20a0 con 0x7f69600d1780 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f6965f41640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69600d39e0 con 0x7f69600b20a0 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f695f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69600d1780 0x7f69600d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f695f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f69600d1780 0x7f69600d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42112/0 (socket says 172.21.15.111:42112) 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.467+0000 7f695f7fe640 1 -- 172.21.15.111:0/1521419321 learned_addr learned my addr 172.21.15.111:0/1521419321 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.468+0000 7f695f7fe640 1 -- 172.21.15.111:0/1521419321 >> v2:172.21.15.192:3300/0 conn(0x7f69600b20a0 msgr2=0x7f69600d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.468+0000 7f6956ffd640 1 --2- 172.21.15.111:0/1521419321 >> v2:172.21.15.192:3300/0 conn(0x7f69600b20a0 0x7f69600d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.468+0000 7f695f7fe640 1 --2- 172.21.15.111:0/1521419321 >> v2:172.21.15.192:3300/0 conn(0x7f69600b20a0 0x7f69600d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.468+0000 7f695f7fe640 1 -- 172.21.15.111:0/1521419321 >> v1:172.21.15.192:6789/0 conn(0x7f69600b2ad0 legacy=0x7f69600ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:15.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.468+0000 7f695f7fe640 1 -- 172.21.15.111:0/1521419321 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600d3cc0 con 0x7f69600d1780 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.468+0000 7f695f7fe640 1 --2- 172.21.15.111:0/1521419321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600d01e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f69600d0c50 tx=0x7f695002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f9976eb2a70f79 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f695effd640 1 -- 172.21.15.111:0/1521419321 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500317e0 con 0x7f69600d1780 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f695effd640 1 -- 172.21.15.111:0/1521419321 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950031940 con 0x7f69600d1780 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f695effd640 1 -- 172.21.15.111:0/1521419321 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950031cd0 con 0x7f69600d1780 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 -- 172.21.15.111:0/1521419321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 msgr2=0x7f69600d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 --2- 172.21.15.111:0/1521419321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600d01e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f69600d0c50 tx=0x7f695002b880 comp rx=0 tx=0).stop 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 -- 172.21.15.111:0/1521419321 shutdown_connections 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 --2- 172.21.15.111:0/1521419321 >> v2:172.21.15.192:3300/0 conn(0x7f69600b20a0 0x7f69600d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 --2- 172.21.15.111:0/1521419321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600d01e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 -- 172.21.15.111:0/1521419321 >> 172.21.15.111:0/1521419321 conn(0x7f696009c4e0 msgr2=0x7f69600d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 -- 172.21.15.111:0/1521419321 shutdown_connections 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.469+0000 7f6965f41640 1 -- 172.21.15.111:0/1521419321 wait complete. 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.470+0000 7f6965f41640 1 Processor -- start 2023-12-04T05:40:15.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 -- start start 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69600b20a0 0x7f696014b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69600b2ad0 0x7f696014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696015fd50 con 0x7f69600d1780 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600b78b0 con 0x7f69600b2ad0 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f6965f41640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696014bb00 con 0x7f69600b20a0 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f695ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f695ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42114/0 (socket says 172.21.15.111:42114) 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.471+0000 7f695ffff640 1 -- 172.21.15.111:0/2439384592 learned_addr learned my addr 172.21.15.111:0/2439384592 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f6956ffd640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69600b2ad0 0x7f696014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695ffff640 1 -- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69600b20a0 msgr2=0x7f696014b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695ffff640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69600b20a0 0x7f696014b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695ffff640 1 -- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69600b2ad0 msgr2=0x7f696014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695ffff640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69600b2ad0 0x7f696014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695ffff640 1 -- 172.21.15.111:0/2439384592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69500094d0 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695ffff640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600b6d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f695800b970 tx=0x7f695800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695800cb40 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695800cca0 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696015f3b0 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958014680 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.472+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696015f970 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.473+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69600693d0 con 0x7f69600d1780 2023-12-04T05:40:15.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.473+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f695801d3b0 con 0x7f69600d1780 2023-12-04T05:40:15.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.474+0000 7f695cff9640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f694006b2a0 0x7f694006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.474+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69580899c0 con 0x7f69600d1780 2023-12-04T05:40:15.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.474+0000 7f695f7fe640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f694006b2a0 0x7f694006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.474+0000 7f695f7fe640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f694006b2a0 0x7f694006d760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6950036000 tx=0x7f6950030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.478+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69580548c0 con 0x7f69600d1780 2023-12-04T05:40:15.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.640+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69600d1240 con 0x7f694006b2a0 2023-12-04T05:40:15.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.647+0000 7f695cff9640 1 -- 172.21.15.111:0/2439384592 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277427 (secure 0 0 0) 0x7f6950031b30 con 0x7f694006b2a0 2023-12-04T05:40:15.651 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:15.652 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:40:15.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.649+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f694006b2a0 msgr2=0x7f694006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.649+0000 7f6965f41640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f694006b2a0 0x7f694006d760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6950036000 tx=0x7f6950030000 comp rx=0 tx=0).stop 2023-12-04T05:40:15.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.649+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 msgr2=0x7f69600b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.649+0000 7f6965f41640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600b6d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f695800b970 tx=0x7f695800be30 comp rx=0 tx=0).stop 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 shutdown_connections 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f694006b2a0 0x7f694006d760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69600d1780 0x7f69600b6d40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69600b2ad0 0x7f696014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 --2- 172.21.15.111:0/2439384592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69600b20a0 0x7f696014b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 >> 172.21.15.111:0/2439384592 conn(0x7f696009c4e0 msgr2=0x7f696006b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 shutdown_connections 2023-12-04T05:40:15.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.650+0000 7f6965f41640 1 -- 172.21.15.111:0/2439384592 wait complete. 2023-12-04T05:40:15.662 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-12-04T05:40:14.483579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219904,"kb_used_data":3700,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754816,"statfs":{"total":773094113280,"available":772868931584,"internally_reserved":0,"allocated":3788800,"data_stored":2157953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.104941"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383186+0000","last_change":"2023-12-04T05:40:09.383186+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383186+0000","last_undegraded":"2023-12-04T05:40:09.383186+0000","last_fullsized":"2023-12-04T05:40:09.383186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382869+0000","last_change":"2023-12-04T05:40:09.382869+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382869+0000","last_undegraded":"2023-12-04T05:40:09.382869+0000","last_fullsized":"2023-12-04T05:40:09.382869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381342+0000","last_change":"2023-12-04T05:40:09.381342+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381342+0000","last_undegraded":"2023-12-04T05:40:09.381342+0000","last_fullsized":"2023-12-04T05:40:09.381342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381105+0000","last_change":"2023-12-04T05:40:09.381105+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381105+0000","last_undegraded":"2023-12-04T05:40:09.381105+0000","last_fullsized":"2023-12-04T05:40:09.381105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380752+0000","last_change":"2023-12-04T05:40:09.380752+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380752+0000","last_undegraded":"2023-12-04T05:40:09.380752+0000","last_fullsized":"2023-12-04T05:40:09.380752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383269+0000","last_change":"2023-12-04T05:40:09.383269+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383269+0000","last_undegraded":"2023-12-04T05:40:09.383269+0000","last_fullsized":"2023-12-04T05:40:09.383269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380659+0000","last_change":"2023-12-04T05:40:09.380659+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380659+0000","last_undegraded":"2023-12-04T05:40:09.380659+0000","last_fullsized":"2023-12-04T05:40:09.380659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382723+0000","last_change":"2023-12-04T05:40:09.382723+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382723+0000","last_undegraded":"2023-12-04T05:40:09.382723+0000","last_fullsized":"2023-12-04T05:40:09.382723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381605+0000","last_change":"2023-12-04T05:40:09.381605+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381605+0000","last_undegraded":"2023-12-04T05:40:09.381605+0000","last_fullsized":"2023-12-04T05:40:09.381605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382799+0000","last_change":"2023-12-04T05:40:09.382799+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382799+0000","last_undegraded":"2023-12-04T05:40:09.382799+0000","last_fullsized":"2023-12-04T05:40:09.382799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380588+0000","last_change":"2023-12-04T05:40:09.380588+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380588+0000","last_undegraded":"2023-12-04T05:40:09.380588+0000","last_fullsized":"2023-12-04T05:40:09.380588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384407+0000","last_change":"2023-12-04T05:40:09.384407+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384407+0000","last_undegraded":"2023-12-04T05:40:09.384407+0000","last_fullsized":"2023-12-04T05:40:09.384407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382706+0000","last_change":"2023-12-04T05:40:09.382706+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382706+0000","last_undegraded":"2023-12-04T05:40:09.382706+0000","last_fullsized":"2023-12-04T05:40:09.382706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383049+0000","last_change":"2023-12-04T05:40:09.383049+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383049+0000","last_undegraded":"2023-12-04T05:40:09.383049+0000","last_fullsized":"2023-12-04T05:40:09.383049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384677+0000","last_change":"2023-12-04T05:40:09.384677+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384677+0000","last_undegraded":"2023-12-04T05:40:09.384677+0000","last_fullsized":"2023-12-04T05:40:09.384677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380488+0000","last_change":"2023-12-04T05:40:09.380488+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380488+0000","last_undegraded":"2023-12-04T05:40:09.380488+0000","last_fullsized":"2023-12-04T05:40:09.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380417+0000","last_change":"2023-12-04T05:40:09.380417+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380417+0000","last_undegraded":"2023-12-04T05:40:09.380417+0000","last_fullsized":"2023-12-04T05:40:09.380417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380404+0000","last_change":"2023-12-04T05:40:09.380404+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380404+0000","last_undegraded":"2023-12-04T05:40:09.380404+0000","last_fullsized":"2023-12-04T05:40:09.380404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383218+0000","last_change":"2023-12-04T05:40:09.383218+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383218+0000","last_undegraded":"2023-12-04T05:40:09.383218+0000","last_fullsized":"2023-12-04T05:40:09.383218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381657+0000","last_change":"2023-12-04T05:40:09.381657+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381657+0000","last_undegraded":"2023-12-04T05:40:09.381657+0000","last_fullsized":"2023-12-04T05:40:09.381657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382052+0000","last_change":"2023-12-04T05:40:09.382052+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382052+0000","last_undegraded":"2023-12-04T05:40:09.382052+0000","last_fullsized":"2023-12-04T05:40:09.382052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384013+0000","last_change":"2023-12-04T05:40:09.384013+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384013+0000","last_undegraded":"2023-12-04T05:40:09.384013+0000","last_fullsized":"2023-12-04T05:40:09.384013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383973+0000","last_change":"2023-12-04T05:40:09.383973+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383973+0000","last_undegraded":"2023-12-04T05:40:09.383973+0000","last_fullsized":"2023-12-04T05:40:09.383973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383973+0000","last_change":"2023-12-04T05:40:09.383973+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383973+0000","last_undegraded":"2023-12-04T05:40:09.383973+0000","last_fullsized":"2023-12-04T05:40:09.383973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380341+0000","last_change":"2023-12-04T05:40:09.380341+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380341+0000","last_undegraded":"2023-12-04T05:40:09.380341+0000","last_fullsized":"2023-12-04T05:40:09.380341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382605+0000","last_change":"2023-12-04T05:40:09.382605+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382605+0000","last_undegraded":"2023-12-04T05:40:09.382605+0000","last_fullsized":"2023-12-04T05:40:09.382605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383277+0000","last_change":"2023-12-04T05:40:09.383277+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383277+0000","last_undegraded":"2023-12-04T05:40:09.383277+0000","last_fullsized":"2023-12-04T05:40:09.383277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-04T05:40:05.524663+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:40:05.524663+0000","last_peered":"2023-12-04T05:40:05.524663+0000","last_clean":"2023-12-04T05:40:05.524663+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:40:05.524663+0000","last_undegraded":"2023-12-04T05:40:05.524663+0000","last_fullsized":"2023-12-04T05:40:05.524663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":711793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.435}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.5}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.629}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.501}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46899999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46400000000000002}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54600000000000004}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.496}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.751}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.315},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26500000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.751}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.253},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.27200000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.307},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.318},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.371}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.39100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.34200000000000003}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.375}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.314},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:40:15.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:40:15.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.800+0000 7f036e361640 1 Processor -- start 2023-12-04T05:40:15.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.800+0000 7f036e361640 1 -- start start 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.800+0000 7f036e361640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f036813cee0 0x7f036806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036e361640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f036815aba0 0x7f036806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036e361640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03680aea20 con 0x7f0368140200 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036e361640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036813d2c0 con 0x7f036815aba0 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036e361640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036815b3a0 con 0x7f036813cee0 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036c8fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f036815aba0 0x7f036806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f03677fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0368140200 0x7f036815a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48806/0 (socket says 172.21.15.111:48806) 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036c8fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f036815aba0 0x7f036806c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42116/0 (socket says 172.21.15.111:42116) 2023-12-04T05:40:15.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f03677fe640 1 -- 172.21.15.111:0/420922206 learned_addr learned my addr 172.21.15.111:0/420922206 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f0367fff640 1 --2- 172.21.15.111:0/420922206 >> v2:172.21.15.192:3300/0 conn(0x7f036813cee0 0x7f036806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036c8fe640 1 -- 172.21.15.111:0/420922206 >> v2:172.21.15.192:3300/0 conn(0x7f036813cee0 msgr2=0x7f036806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036c8fe640 1 --2- 172.21.15.111:0/420922206 >> v2:172.21.15.192:3300/0 conn(0x7f036813cee0 0x7f036806c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036c8fe640 1 -- 172.21.15.111:0/420922206 >> v1:172.21.15.111:6789/0 conn(0x7f0368140200 legacy=0x7f036815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f036c8fe640 1 -- 172.21.15.111:0/420922206 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036815b0c0 con 0x7f036815aba0 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.801+0000 7f0366ffd640 1 -- 172.21.15.111:0/420922206 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025821693 0 0) 0x7f03680aea20 con 0x7f0368140200 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036c8fe640 1 --2- 172.21.15.111:0/420922206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f036815aba0 0x7f036806c790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03680b2280 tx=0x7f036002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2252320ffb614f21 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f0366ffd640 1 -- 172.21.15.111:0/420922206 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036002e9d0 con 0x7f036815aba0 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f0366ffd640 1 -- 172.21.15.111:0/420922206 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036002eb30 con 0x7f036815aba0 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f0366ffd640 1 -- 172.21.15.111:0/420922206 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360009540 con 0x7f036815aba0 2023-12-04T05:40:15.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 -- 172.21.15.111:0/420922206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f036815aba0 msgr2=0x7f036806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 --2- 172.21.15.111:0/420922206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f036815aba0 0x7f036806c790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03680b2280 tx=0x7f036002e030 comp rx=0 tx=0).stop 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 -- 172.21.15.111:0/420922206 shutdown_connections 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 --2- 172.21.15.111:0/420922206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f036815aba0 0x7f036806c790 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 --2- 172.21.15.111:0/420922206 >> v2:172.21.15.192:3300/0 conn(0x7f036813cee0 0x7f036806c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 -- 172.21.15.111:0/420922206 >> 172.21.15.111:0/420922206 conn(0x7f036806a4c0 msgr2=0x7f036806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.802+0000 7f036e361640 1 -- 172.21.15.111:0/420922206 shutdown_connections 2023-12-04T05:40:15.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.803+0000 7f036e361640 1 -- 172.21.15.111:0/420922206 wait complete. 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 Processor -- start 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 -- start start 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036813cee0 0x7f03681f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 0x7f03680cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036815aba0 0x7f03680cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f36a0 con 0x7f0368140200 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f2d90 con 0x7f036813cee0 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.804+0000 7f036e361640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f70b0 con 0x7f036815aba0 2023-12-04T05:40:15.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 0x7f03680cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 0x7f03680cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42132/0 (socket says 172.21.15.111:42132) 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 -- 172.21.15.111:0/1229356026 learned_addr learned my addr 172.21.15.111:0/1229356026 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f0367fff640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036813cee0 0x7f03681f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f036c8fe640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036815aba0 0x7f03680cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 -- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036815aba0 msgr2=0x7f03680cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036815aba0 0x7f03680cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 -- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036813cee0 msgr2=0x7f03681f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036813cee0 0x7f03681f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 -- 172.21.15.111:0/1229356026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036000bc80 con 0x7f0368140200 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f0367fff640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036813cee0 0x7f03681f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f03677fe640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 0x7f03680cc1a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c4d0 tx=0x7f035c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0074e0 con 0x7f0368140200 2023-12-04T05:40:15.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c007660 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03681f3050 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c004a20 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.805+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03681f05d0 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.806+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03681f0790 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.806+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f035c004b80 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.807+0000 7f0364ff9640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f035406b2a0 0x7f035406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.807+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f035c089070 con 0x7f0368140200 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.807+0000 7f0367fff640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f035406b2a0 0x7f035406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:15.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.807+0000 7f0367fff640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f035406b2a0 0x7f035406d760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0358006fd0 tx=0x7f0358008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:15.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.811+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f035c053f70 con 0x7f0368140200 2023-12-04T05:40:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.974+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f036813fce0 con 0x7f035406b2a0 2023-12-04T05:40:15.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.981+0000 7f0364ff9640 1 -- 172.21.15.111:0/1229356026 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277427 (secure 0 0 0) 0x7f03580095f0 con 0x7f035406b2a0 2023-12-04T05:40:15.985 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:15.987 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:40:15.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f035406b2a0 msgr2=0x7f035406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f035406b2a0 0x7f035406d760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0358006fd0 tx=0x7f0358008040 comp rx=0 tx=0).stop 2023-12-04T05:40:15.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 msgr2=0x7f03680cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 0x7f03680cc1a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c4d0 tx=0x7f035c00c990 comp rx=0 tx=0).stop 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 shutdown_connections 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f035406b2a0 0x7f035406d760 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036815aba0 0x7f03680cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0368140200 0x7f03680cc1a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 --2- 172.21.15.111:0/1229356026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036813cee0 0x7f03681f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 >> 172.21.15.111:0/1229356026 conn(0x7f036806a4c0 msgr2=0x7f036814fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 shutdown_connections 2023-12-04T05:40:15.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:15.984+0000 7f036e361640 1 -- 172.21.15.111:0/1229356026 wait complete. 2023-12-04T05:40:15.996 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-12-04T05:40:14.483579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219904,"kb_used_data":3700,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754816,"statfs":{"total":773094113280,"available":772868931584,"internally_reserved":0,"allocated":3788800,"data_stored":2157953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.104941"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383186+0000","last_change":"2023-12-04T05:40:09.383186+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383186+0000","last_undegraded":"2023-12-04T05:40:09.383186+0000","last_fullsized":"2023-12-04T05:40:09.383186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382869+0000","last_change":"2023-12-04T05:40:09.382869+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382869+0000","last_undegraded":"2023-12-04T05:40:09.382869+0000","last_fullsized":"2023-12-04T05:40:09.382869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381342+0000","last_change":"2023-12-04T05:40:09.381342+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381342+0000","last_undegraded":"2023-12-04T05:40:09.381342+0000","last_fullsized":"2023-12-04T05:40:09.381342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381105+0000","last_change":"2023-12-04T05:40:09.381105+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381105+0000","last_undegraded":"2023-12-04T05:40:09.381105+0000","last_fullsized":"2023-12-04T05:40:09.381105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380752+0000","last_change":"2023-12-04T05:40:09.380752+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380752+0000","last_undegraded":"2023-12-04T05:40:09.380752+0000","last_fullsized":"2023-12-04T05:40:09.380752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383269+0000","last_change":"2023-12-04T05:40:09.383269+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383269+0000","last_undegraded":"2023-12-04T05:40:09.383269+0000","last_fullsized":"2023-12-04T05:40:09.383269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380659+0000","last_change":"2023-12-04T05:40:09.380659+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380659+0000","last_undegraded":"2023-12-04T05:40:09.380659+0000","last_fullsized":"2023-12-04T05:40:09.380659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382723+0000","last_change":"2023-12-04T05:40:09.382723+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382723+0000","last_undegraded":"2023-12-04T05:40:09.382723+0000","last_fullsized":"2023-12-04T05:40:09.382723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381605+0000","last_change":"2023-12-04T05:40:09.381605+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381605+0000","last_undegraded":"2023-12-04T05:40:09.381605+0000","last_fullsized":"2023-12-04T05:40:09.381605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382799+0000","last_change":"2023-12-04T05:40:09.382799+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382799+0000","last_undegraded":"2023-12-04T05:40:09.382799+0000","last_fullsized":"2023-12-04T05:40:09.382799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380588+0000","last_change":"2023-12-04T05:40:09.380588+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380588+0000","last_undegraded":"2023-12-04T05:40:09.380588+0000","last_fullsized":"2023-12-04T05:40:09.380588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384407+0000","last_change":"2023-12-04T05:40:09.384407+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384407+0000","last_undegraded":"2023-12-04T05:40:09.384407+0000","last_fullsized":"2023-12-04T05:40:09.384407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382706+0000","last_change":"2023-12-04T05:40:09.382706+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382706+0000","last_undegraded":"2023-12-04T05:40:09.382706+0000","last_fullsized":"2023-12-04T05:40:09.382706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383049+0000","last_change":"2023-12-04T05:40:09.383049+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383049+0000","last_undegraded":"2023-12-04T05:40:09.383049+0000","last_fullsized":"2023-12-04T05:40:09.383049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384677+0000","last_change":"2023-12-04T05:40:09.384677+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384677+0000","last_undegraded":"2023-12-04T05:40:09.384677+0000","last_fullsized":"2023-12-04T05:40:09.384677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380488+0000","last_change":"2023-12-04T05:40:09.380488+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380488+0000","last_undegraded":"2023-12-04T05:40:09.380488+0000","last_fullsized":"2023-12-04T05:40:09.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380417+0000","last_change":"2023-12-04T05:40:09.380417+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380417+0000","last_undegraded":"2023-12-04T05:40:09.380417+0000","last_fullsized":"2023-12-04T05:40:09.380417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380404+0000","last_change":"2023-12-04T05:40:09.380404+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380404+0000","last_undegraded":"2023-12-04T05:40:09.380404+0000","last_fullsized":"2023-12-04T05:40:09.380404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383218+0000","last_change":"2023-12-04T05:40:09.383218+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383218+0000","last_undegraded":"2023-12-04T05:40:09.383218+0000","last_fullsized":"2023-12-04T05:40:09.383218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.381657+0000","last_change":"2023-12-04T05:40:09.381657+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.381657+0000","last_undegraded":"2023-12-04T05:40:09.381657+0000","last_fullsized":"2023-12-04T05:40:09.381657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382052+0000","last_change":"2023-12-04T05:40:09.382052+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382052+0000","last_undegraded":"2023-12-04T05:40:09.382052+0000","last_fullsized":"2023-12-04T05:40:09.382052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.384013+0000","last_change":"2023-12-04T05:40:09.384013+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.384013+0000","last_undegraded":"2023-12-04T05:40:09.384013+0000","last_fullsized":"2023-12-04T05:40:09.384013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383973+0000","last_change":"2023-12-04T05:40:09.383973+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383973+0000","last_undegraded":"2023-12-04T05:40:09.383973+0000","last_fullsized":"2023-12-04T05:40:09.383973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:09.373629+0000","last_change":"2023-12-04T05:40:09.373629+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.373629+0000","last_undegraded":"2023-12-04T05:40:09.373629+0000","last_fullsized":"2023-12-04T05:40:09.373629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383973+0000","last_change":"2023-12-04T05:40:09.383973+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383973+0000","last_undegraded":"2023-12-04T05:40:09.383973+0000","last_fullsized":"2023-12-04T05:40:09.383973+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.380341+0000","last_change":"2023-12-04T05:40:09.380341+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.380341+0000","last_undegraded":"2023-12-04T05:40:09.380341+0000","last_fullsized":"2023-12-04T05:40:09.380341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.382605+0000","last_change":"2023-12-04T05:40:09.382605+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.382605+0000","last_undegraded":"2023-12-04T05:40:09.382605+0000","last_fullsized":"2023-12-04T05:40:09.382605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:40:10.377744+0000","last_change":"2023-12-04T05:40:10.377744+0000","last_active":"2023-12-04T05:40:10.377744+0000","last_peered":"2023-12-04T05:40:10.377744+0000","last_clean":"2023-12-04T05:40:10.377744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:10.377744+0000","last_undegraded":"2023-12-04T05:40:10.377744+0000","last_fullsized":"2023-12-04T05:40:10.377744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-12-04T05:40:09.383277+0000","last_change":"2023-12-04T05:40:09.383277+0000","last_active":"2023-12-04T05:40:09.373629+0000","last_peered":"2023-12-04T05:40:09.373629+0000","last_clean":"2023-12-04T05:40:09.373629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:40:09.383277+0000","last_undegraded":"2023-12-04T05:40:09.383277+0000","last_fullsized":"2023-12-04T05:40:09.383277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-04T05:40:05.524663+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:40:05.524663+0000","last_peered":"2023-12-04T05:40:05.524663+0000","last_clean":"2023-12-04T05:40:05.524663+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:40:05.524663+0000","last_undegraded":"2023-12-04T05:40:05.524663+0000","last_fullsized":"2023-12-04T05:40:05.524663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":711793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.435}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.5}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.629}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.501}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46899999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46400000000000002}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55400000000000005}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54600000000000004}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.496}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.751}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.315},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26500000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.751}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.253},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.27200000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.307},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.628},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":121425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.318},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.371}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.39100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40600000000000003}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.34200000000000003}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.375}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.314},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-12-04T05:40:16.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-12-04T05:40:16.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-12-04T05:40:16.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-12-04T05:40:16.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-12-04T05:40:16.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383186+0000', 'last_change': '2023-12-04T05:40:09.383186+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383186+0000', 'last_undegraded': '2023-12-04T05:40:09.383186+0000', 'last_fullsized': '2023-12-04T05:40:09.383186+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:09.373629+0000', 'last_change': '2023-12-04T05:40:09.373629+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.373629+0000', 'last_undegraded': '2023-12-04T05:40:09.373629+0000', 'last_fullsized': '2023-12-04T05:40:09.373629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.382869+0000', 'last_change': '2023-12-04T05:40:09.382869+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.382869+0000', 'last_undegraded': '2023-12-04T05:40:09.382869+0000', 'last_fullsized': '2023-12-04T05:40:09.382869+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.381342+0000', 'last_change': '2023-12-04T05:40:09.381342+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.381342+0000', 'last_undegraded': '2023-12-04T05:40:09.381342+0000', 'last_fullsized': '2023-12-04T05:40:09.381342+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.381105+0000', 'last_change': '2023-12-04T05:40:09.381105+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.381105+0000', 'last_undegraded': '2023-12-04T05:40:09.381105+0000', 'last_fullsized': '2023-12-04T05:40:09.381105+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-12-04T05:40:16.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380752+0000', 'last_change': '2023-12-04T05:40:09.380752+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380752+0000', 'last_undegraded': '2023-12-04T05:40:09.380752+0000', 'last_fullsized': '2023-12-04T05:40:09.380752+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383269+0000', 'last_change': '2023-12-04T05:40:09.383269+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383269+0000', 'last_undegraded': '2023-12-04T05:40:09.383269+0000', 'last_fullsized': '2023-12-04T05:40:09.383269+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-12-04T05:40:16.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380659+0000', 'last_change': '2023-12-04T05:40:09.380659+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380659+0000', 'last_undegraded': '2023-12-04T05:40:09.380659+0000', 'last_fullsized': '2023-12-04T05:40:09.380659+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.382723+0000', 'last_change': '2023-12-04T05:40:09.382723+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.382723+0000', 'last_undegraded': '2023-12-04T05:40:09.382723+0000', 'last_fullsized': '2023-12-04T05:40:09.382723+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:09.373629+0000', 'last_change': '2023-12-04T05:40:09.373629+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.373629+0000', 'last_undegraded': '2023-12-04T05:40:09.373629+0000', 'last_fullsized': '2023-12-04T05:40:09.373629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-12-04T05:40:16.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.381605+0000', 'last_change': '2023-12-04T05:40:09.381605+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.381605+0000', 'last_undegraded': '2023-12-04T05:40:09.381605+0000', 'last_fullsized': '2023-12-04T05:40:09.381605+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-04T05:40:16.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-12-04T05:40:16.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-12-04T05:40:16.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.382799+0000', 'last_change': '2023-12-04T05:40:09.382799+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.382799+0000', 'last_undegraded': '2023-12-04T05:40:09.382799+0000', 'last_fullsized': '2023-12-04T05:40:09.382799+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-12-04T05:40:16.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380588+0000', 'last_change': '2023-12-04T05:40:09.380588+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380588+0000', 'last_undegraded': '2023-12-04T05:40:09.380588+0000', 'last_fullsized': '2023-12-04T05:40:09.380588+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.384407+0000', 'last_change': '2023-12-04T05:40:09.384407+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.384407+0000', 'last_undegraded': '2023-12-04T05:40:09.384407+0000', 'last_fullsized': '2023-12-04T05:40:09.384407+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.382706+0000', 'last_change': '2023-12-04T05:40:09.382706+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.382706+0000', 'last_undegraded': '2023-12-04T05:40:09.382706+0000', 'last_fullsized': '2023-12-04T05:40:09.382706+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383049+0000', 'last_change': '2023-12-04T05:40:09.383049+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383049+0000', 'last_undegraded': '2023-12-04T05:40:09.383049+0000', 'last_fullsized': '2023-12-04T05:40:09.383049+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.384677+0000', 'last_change': '2023-12-04T05:40:09.384677+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.384677+0000', 'last_undegraded': '2023-12-04T05:40:09.384677+0000', 'last_fullsized': '2023-12-04T05:40:09.384677+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-12-04T05:40:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380488+0000', 'last_change': '2023-12-04T05:40:09.380488+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380488+0000', 'last_undegraded': '2023-12-04T05:40:09.380488+0000', 'last_fullsized': '2023-12-04T05:40:09.380488+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380417+0000', 'last_change': '2023-12-04T05:40:09.380417+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380417+0000', 'last_undegraded': '2023-12-04T05:40:09.380417+0000', 'last_fullsized': '2023-12-04T05:40:09.380417+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380404+0000', 'last_change': '2023-12-04T05:40:09.380404+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380404+0000', 'last_undegraded': '2023-12-04T05:40:09.380404+0000', 'last_fullsized': '2023-12-04T05:40:09.380404+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-12-04T05:40:16.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:09.373629+0000', 'last_change': '2023-12-04T05:40:09.373629+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.373629+0000', 'last_undegraded': '2023-12-04T05:40:09.373629+0000', 'last_fullsized': '2023-12-04T05:40:09.373629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383218+0000', 'last_change': '2023-12-04T05:40:09.383218+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383218+0000', 'last_undegraded': '2023-12-04T05:40:09.383218+0000', 'last_fullsized': '2023-12-04T05:40:09.383218+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.381657+0000', 'last_change': '2023-12-04T05:40:09.381657+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.381657+0000', 'last_undegraded': '2023-12-04T05:40:09.381657+0000', 'last_fullsized': '2023-12-04T05:40:09.381657+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-12-04T05:40:16.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.382052+0000', 'last_change': '2023-12-04T05:40:09.382052+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.382052+0000', 'last_undegraded': '2023-12-04T05:40:09.382052+0000', 'last_fullsized': '2023-12-04T05:40:09.382052+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.384013+0000', 'last_change': '2023-12-04T05:40:09.384013+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.384013+0000', 'last_undegraded': '2023-12-04T05:40:09.384013+0000', 'last_fullsized': '2023-12-04T05:40:09.384013+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-12-04T05:40:16.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:09.373629+0000', 'last_change': '2023-12-04T05:40:09.373629+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.373629+0000', 'last_undegraded': '2023-12-04T05:40:09.373629+0000', 'last_fullsized': '2023-12-04T05:40:09.373629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383973+0000', 'last_change': '2023-12-04T05:40:09.383973+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383973+0000', 'last_undegraded': '2023-12-04T05:40:09.383973+0000', 'last_fullsized': '2023-12-04T05:40:09.383973+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:09.373629+0000', 'last_change': '2023-12-04T05:40:09.373629+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.373629+0000', 'last_undegraded': '2023-12-04T05:40:09.373629+0000', 'last_fullsized': '2023-12-04T05:40:09.373629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-12-04T05:40:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383973+0000', 'last_change': '2023-12-04T05:40:09.383973+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383973+0000', 'last_undegraded': '2023-12-04T05:40:09.383973+0000', 'last_fullsized': '2023-12-04T05:40:09.383973+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.380341+0000', 'last_change': '2023-12-04T05:40:09.380341+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.380341+0000', 'last_undegraded': '2023-12-04T05:40:09.380341+0000', 'last_fullsized': '2023-12-04T05:40:09.380341+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.382605+0000', 'last_change': '2023-12-04T05:40:09.382605+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.382605+0000', 'last_undegraded': '2023-12-04T05:40:09.382605+0000', 'last_fullsized': '2023-12-04T05:40:09.382605+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-04T05:40:10.377744+0000', 'last_change': '2023-12-04T05:40:10.377744+0000', 'last_active': '2023-12-04T05:40:10.377744+0000', 'last_peered': '2023-12-04T05:40:10.377744+0000', 'last_clean': '2023-12-04T05:40:10.377744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:10.377744+0000', 'last_undegraded': '2023-12-04T05:40:10.377744+0000', 'last_fullsized': '2023-12-04T05:40:10.377744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:10.377744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-04T05:40:16.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-12-04T05:40:16.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-12-04T05:40:09.383277+0000', 'last_change': '2023-12-04T05:40:09.383277+0000', 'last_active': '2023-12-04T05:40:09.373629+0000', 'last_peered': '2023-12-04T05:40:09.373629+0000', 'last_clean': '2023-12-04T05:40:09.373629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-04T05:40:09.383277+0000', 'last_undegraded': '2023-12-04T05:40:09.383277+0000', 'last_fullsized': '2023-12-04T05:40:09.383277+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'last_clean_scrub_stamp': '2023-12-04T05:40:09.373629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-04T05:40:16.022 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-04T05:40:16.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.133+0000 7fc262301640 1 Processor -- start 2023-12-04T05:40:16.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.133+0000 7fc262301640 1 -- start start 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc262301640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc25c15c060 0x7fc25c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc262301640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc25c06b4d0 0x7fc25c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc262301640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0cd240 con 0x7fc25c151890 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc262301640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c06bec0 con 0x7fc25c06b4d0 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc262301640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c0cd520 con 0x7fc25c15c060 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc25c06b4d0 0x7fc25c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc25c06b4d0 0x7fc25c06b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42140/0 (socket says 172.21.15.111:42140) 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 -- 172.21.15.111:0/1373992555 learned_addr learned my addr 172.21.15.111:0/1373992555 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25bfff640 1 --2- 172.21.15.111:0/1373992555 >> v2:172.21.15.192:3300/0 conn(0x7fc25c15c060 0x7fc25c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 -- 172.21.15.111:0/1373992555 >> v2:172.21.15.192:3300/0 conn(0x7fc25c15c060 msgr2=0x7fc25c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:16.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 --2- 172.21.15.111:0/1373992555 >> v2:172.21.15.192:3300/0 conn(0x7fc25c15c060 0x7fc25c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 -- 172.21.15.111:0/1373992555 >> v1:172.21.15.192:6789/0 conn(0x7fc25c151890 legacy=0x7fc25c151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.134+0000 7fc25b7fe640 1 -- 172.21.15.111:0/1373992555 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0cd800 con 0x7fc25c06b4d0 2023-12-04T05:40:16.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.135+0000 7fc25b7fe640 1 --2- 172.21.15.111:0/1373992555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c06b4d0 0x7fc25c06b980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc25c071520 tx=0x7fc25002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794da08d499a9613 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:16.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.135+0000 7fc25affd640 1 -- 172.21.15.111:0/1373992555 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250031870 con 0x7fc25c06b4d0 2023-12-04T05:40:16.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.135+0000 7fc25affd640 1 -- 172.21.15.111:0/1373992555 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2500319d0 con 0x7fc25c06b4d0 2023-12-04T05:40:16.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.135+0000 7fc25affd640 1 -- 172.21.15.111:0/1373992555 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250031d60 con 0x7fc25c06b4d0 2023-12-04T05:40:16.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 -- 172.21.15.111:0/1373992555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c06b4d0 msgr2=0x7fc25c06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 --2- 172.21.15.111:0/1373992555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c06b4d0 0x7fc25c06b980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc25c071520 tx=0x7fc25002b910 comp rx=0 tx=0).stop 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 -- 172.21.15.111:0/1373992555 shutdown_connections 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 --2- 172.21.15.111:0/1373992555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c06b4d0 0x7fc25c06b980 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 --2- 172.21.15.111:0/1373992555 >> v2:172.21.15.192:3300/0 conn(0x7fc25c15c060 0x7fc25c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 -- 172.21.15.111:0/1373992555 >> 172.21.15.111:0/1373992555 conn(0x7fc25c0b1b50 msgr2=0x7fc25c070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 -- 172.21.15.111:0/1373992555 shutdown_connections 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.136+0000 7fc262301640 1 -- 172.21.15.111:0/1373992555 wait complete. 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.137+0000 7fc262301640 1 Processor -- start 2023-12-04T05:40:16.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.137+0000 7fc262301640 1 -- start start 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc262301640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc25c06b4d0 0x7fc25c1f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc262301640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 0x7fc25c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc262301640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc25c15c060 0x7fc25c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc262301640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1f5100 con 0x7fc25c151890 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc262301640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1f1b70 con 0x7fc25c15c060 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc262301640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1f0f60 con 0x7fc25c06b4d0 2023-12-04T05:40:16.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 0x7fc25c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 0x7fc25c06d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42142/0 (socket says 172.21.15.111:42142) 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 -- 172.21.15.111:0/3609318672 learned_addr learned my addr 172.21.15.111:0/3609318672 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 -- 172.21.15.111:0/3609318672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc25c06b4d0 msgr2=0x7fc25c1f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc25c06b4d0 0x7fc25c1f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 -- 172.21.15.111:0/3609318672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc25c15c060 msgr2=0x7fc25c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc25c15c060 0x7fc25c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 -- 172.21.15.111:0/3609318672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500094d0 con 0x7fc25c151890 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.138+0000 7fc25b7fe640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 0x7fc25c06d240 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc25002b950 tx=0x7fc2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.139+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25003abe0 con 0x7fc25c151890 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.139+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25003ad60 con 0x7fc25c151890 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.139+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c1f1240 con 0x7fc25c151890 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.139+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250044630 con 0x7fc25c151890 2023-12-04T05:40:16.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.139+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c1f1800 con 0x7fc25c151890 2023-12-04T05:40:16.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.140+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c0693d0 con 0x7fc25c151890 2023-12-04T05:40:16.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.140+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc25004d450 con 0x7fc25c151890 2023-12-04T05:40:16.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.140+0000 7fc258ff9640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc24806b2d0 0x7fc24806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.140+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc25002d040 con 0x7fc25c151890 2023-12-04T05:40:16.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.141+0000 7fc25bfff640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc24806b2d0 0x7fc24806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.141+0000 7fc25bfff640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc24806b2d0 0x7fc24806d790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006a90 tx=0x7fc24c006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:16.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.145+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc250076f30 con 0x7fc25c151890 2023-12-04T05:40:16.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:40:16.367 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:40:16.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.374+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc25c0717f0 con 0x7fc25c151890 2023-12-04T05:40:16.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.374+0000 7fc258ff9640 1 -- 172.21.15.111:0/3609318672 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1655 (secure 0 0 0) 0x7fc250035690 con 0x7fc25c151890 2023-12-04T05:40:16.376 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:16.376 INFO:teuthology.orchestra.run.smithi111.stdout:{"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1701668343,"num_in_osds":8,"osd_in_since":1701668334,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":69},{"state_name":"creating+peering","count":27},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225181696,"bytes_avail":772868931584,"bytes_total":773094113280,"unknown_pgs_ratio":0.71134018898010254,"inactive_pgs_ratio":0.27835050225257874},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5235}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-12-04T05:40:06.482385+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-04T05:40:16.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc24806b2d0 msgr2=0x7fc24806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc24806b2d0 0x7fc24806d790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006a90 tx=0x7fc24c006a20 comp rx=0 tx=0).stop 2023-12-04T05:40:16.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 msgr2=0x7fc25c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 0x7fc25c06d240 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc25002b950 tx=0x7fc2500046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 shutdown_connections 2023-12-04T05:40:16.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc24806b2d0 0x7fc24806d790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc25c15c060 0x7fc25c13f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc25c151890 0x7fc25c06d240 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 --2- 172.21.15.111:0/3609318672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc25c06b4d0 0x7fc25c1f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.376+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 >> 172.21.15.111:0/3609318672 conn(0x7fc25c0b1b50 msgr2=0x7fc25c151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:16.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.377+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 shutdown_connections 2023-12-04T05:40:16.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.377+0000 7fc262301640 1 -- 172.21.15.111:0/3609318672 wait complete. 2023-12-04T05:40:16.389 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-04T05:40:16.389 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:40:16.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc3a815640 1 Processor -- start 2023-12-04T05:40:16.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc3a815640 1 -- start start 2023-12-04T05:40:16.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc3a815640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 0x7fdc34164e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc3a815640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc340aea20 con 0x7fdc3413cee0 2023-12-04T05:40:16.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc3a815640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3413d2c0 con 0x7fdc3406c960 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc3a815640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3415b3a0 con 0x7fdc3415aba0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc33fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdc3413cee0 0x7fdc3406c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48816/0 (socket says 172.21.15.111:48816) 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.528+0000 7fdc33fff640 1 -- 172.21.15.111:0/922705102 learned_addr learned my addr 172.21.15.111:0/922705102 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 0x7fdc34164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163060383 0 0) 0x7fdc340aea20 con 0x7fdc3413cee0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc1c003520 con 0x7fdc3413cee0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2784595650 0 0) 0x7fdc1c003520 con 0x7fdc3413cee0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 msgr2=0x7fdc34164e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 0x7fdc34164e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 >> v1:172.21.15.192:6789/0 conn(0x7fdc3406c960 legacy=0x7fdc3415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3415b0c0 con 0x7fdc3413cee0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fdc28001670 con 0x7fdc3413cee0 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 >> v1:172.21.15.111:6789/0 conn(0x7fdc3413cee0 legacy=0x7fdc3406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0039a0 0x7fdc1c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 0x7fdc1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc1c004d00 0x7fdc1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3415b3a0 con 0x7fdc1c004d00 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00ba30 con 0x7fdc1c0039a0 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00bbf0 con 0x7fdc3415aba0 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc1c004d00 0x7fdc1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc33fff640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0039a0 0x7fdc1c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc337fe640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 0x7fdc1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 -- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 msgr2=0x7fdc1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc3415aba0 0x7fdc1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 -- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0039a0 msgr2=0x7fdc1c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0039a0 0x7fdc1c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.529+0000 7fdc38db2640 1 -- 172.21.15.111:0/922705102 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c003520 con 0x7fdc1c004d00 2023-12-04T05:40:16.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.530+0000 7fdc38db2640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc1c004d00 0x7fdc1c005180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc3415aff0 tx=0x7fdc1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c306b9db511022d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.530+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c02cb90 con 0x7fdc1c004d00 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.530+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c00f040 con 0x7fdc1c004d00 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.530+0000 7fdc32ffd640 1 -- 172.21.15.111:0/922705102 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c034440 con 0x7fdc1c004d00 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc3a815640 1 -- 172.21.15.111:0/922705102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc1c004d00 msgr2=0x7fdc1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc3a815640 1 --2- 172.21.15.111:0/922705102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc1c004d00 0x7fdc1c005180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc3415aff0 tx=0x7fdc1c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc38db2640 1 -- 172.21.15.111:0/922705102 reap_dead start 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc3a815640 1 -- 172.21.15.111:0/922705102 shutdown_connections 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc3a815640 1 -- 172.21.15.111:0/922705102 >> 172.21.15.111:0/922705102 conn(0x7fdc3406a4c0 msgr2=0x7fdc3406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc3a815640 1 -- 172.21.15.111:0/922705102 shutdown_connections 2023-12-04T05:40:16.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.531+0000 7fdc3a815640 1 -- 172.21.15.111:0/922705102 wait complete. 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.532+0000 7fdc3a815640 1 Processor -- start 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.532+0000 7fdc3a815640 1 -- start start 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc3a815640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c004d00 0x7fdc34134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc3a815640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc34137610 0x7fdc340ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc3a815640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 0x7fdc340ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc3a815640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34152de0 con 0x7fdc34071c10 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc3a815640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc341524d0 con 0x7fdc34137610 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc3a815640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34151750 con 0x7fdc1c004d00 2023-12-04T05:40:16.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 0x7fdc340ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 0x7fdc340ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42158/0 (socket says 172.21.15.111:42158) 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 -- 172.21.15.111:0/3432696233 learned_addr learned my addr 172.21.15.111:0/3432696233 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc337fe640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc34137610 0x7fdc340ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc33fff640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c004d00 0x7fdc34134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 -- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c004d00 msgr2=0x7fdc34134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c004d00 0x7fdc34134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 -- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc34137610 msgr2=0x7fdc340ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc34137610 0x7fdc340ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 -- 172.21.15.111:0/3432696233 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34152720 con 0x7fdc34071c10 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.533+0000 7fdc38db2640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 0x7fdc340ce5b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c037040 tx=0x7fdc2c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.534+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c003990 con 0x7fdc34071c10 2023-12-04T05:40:16.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.534+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c003af0 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.534+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc34152a00 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.534+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c003ea0 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.534+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc34153200 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.535+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc34152f20 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.535+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fdc2c034940 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.536+0000 7fdc30ff9640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdc2006b2d0 0x7fdc2006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.536+0000 7fdc33fff640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdc2006b2d0 0x7fdc2006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.536+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc2c0ab9b0 con 0x7fdc34071c10 2023-12-04T05:40:16.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.536+0000 7fdc33fff640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdc2006b2d0 0x7fdc2006d790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc28000c00 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:16.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.540+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc34152f20 con 0x7fdc34071c10 2023-12-04T05:40:16.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.703+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc3413fce0 con 0x7fdc2006b2d0 2023-12-04T05:40:16.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.710+0000 7fdc30ff9640 1 -- 172.21.15.111:0/3432696233 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290025 (secure 0 0 0) 0x7fdc28007750 con 0x7fdc2006b2d0 2023-12-04T05:40:16.714 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:16.715 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdc2006b2d0 msgr2=0x7fdc2006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdc2006b2d0 0x7fdc2006d790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc28000c00 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 msgr2=0x7fdc340ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 0x7fdc340ce5b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c037040 tx=0x7fdc2c009750 comp rx=0 tx=0).stop 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 shutdown_connections 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdc2006b2d0 0x7fdc2006d790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc34071c10 0x7fdc340ce5b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc34137610 0x7fdc340ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 --2- 172.21.15.111:0/3432696233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c004d00 0x7fdc34134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:16.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 >> 172.21.15.111:0/3432696233 conn(0x7fdc3406a4c0 msgr2=0x7fdc34071500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:16.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 shutdown_connections 2023-12-04T05:40:16.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:16.713+0000 7fdc3a815640 1 -- 172.21.15.111:0/3432696233 wait complete. 2023-12-04T05:40:16.725 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-12-04T05:40:16.483900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.103187"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225242+0000","last_change":"2023-12-04T05:40:11.422578+0000","last_active":"2023-12-04T05:40:13.225242+0000","last_peered":"2023-12-04T05:40:13.225242+0000","last_clean":"2023-12-04T05:40:13.225242+0000","last_became_active":"2023-12-04T05:40:11.422151+0000","last_became_peered":"2023-12-04T05:40:11.422151+0000","last_unstale":"2023-12-04T05:40:13.225242+0000","last_undegraded":"2023-12-04T05:40:13.225242+0000","last_fullsized":"2023-12-04T05:40:13.225242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:14.845449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326842+0000","last_change":"2023-12-04T05:40:11.407146+0000","last_active":"2023-12-04T05:40:13.326842+0000","last_peered":"2023-12-04T05:40:13.326842+0000","last_clean":"2023-12-04T05:40:13.326842+0000","last_became_active":"2023-12-04T05:40:11.406358+0000","last_became_peered":"2023-12-04T05:40:11.406358+0000","last_unstale":"2023-12-04T05:40:13.326842+0000","last_undegraded":"2023-12-04T05:40:13.326842+0000","last_fullsized":"2023-12-04T05:40:13.326842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:47:31.152090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400287+0000","last_change":"2023-12-04T05:40:11.407620+0000","last_active":"2023-12-04T05:40:12.400287+0000","last_peered":"2023-12-04T05:40:12.400287+0000","last_clean":"2023-12-04T05:40:12.400287+0000","last_became_active":"2023-12-04T05:40:11.407370+0000","last_became_peered":"2023-12-04T05:40:11.407370+0000","last_unstale":"2023-12-04T05:40:12.400287+0000","last_undegraded":"2023-12-04T05:40:12.400287+0000","last_fullsized":"2023-12-04T05:40:12.400287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:44:28.365470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025333+0000","last_change":"2023-12-04T05:40:11.415373+0000","last_active":"2023-12-04T05:40:13.025333+0000","last_peered":"2023-12-04T05:40:13.025333+0000","last_clean":"2023-12-04T05:40:13.025333+0000","last_became_active":"2023-12-04T05:40:11.415117+0000","last_became_peered":"2023-12-04T05:40:11.415117+0000","last_unstale":"2023-12-04T05:40:13.025333+0000","last_undegraded":"2023-12-04T05:40:13.025333+0000","last_fullsized":"2023-12-04T05:40:13.025333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.327031+0000","last_change":"2023-12-04T05:40:11.407975+0000","last_active":"2023-12-04T05:40:13.327031+0000","last_peered":"2023-12-04T05:40:13.327031+0000","last_clean":"2023-12-04T05:40:13.327031+0000","last_became_active":"2023-12-04T05:40:11.407721+0000","last_became_peered":"2023-12-04T05:40:11.407721+0000","last_unstale":"2023-12-04T05:40:13.327031+0000","last_undegraded":"2023-12-04T05:40:13.327031+0000","last_fullsized":"2023-12-04T05:40:13.327031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:52.381604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326750+0000","last_change":"2023-12-04T05:40:11.396951+0000","last_active":"2023-12-04T05:40:13.326750+0000","last_peered":"2023-12-04T05:40:13.326750+0000","last_clean":"2023-12-04T05:40:13.326750+0000","last_became_active":"2023-12-04T05:40:11.396700+0000","last_became_peered":"2023-12-04T05:40:11.396700+0000","last_unstale":"2023-12-04T05:40:13.326750+0000","last_undegraded":"2023-12-04T05:40:13.326750+0000","last_fullsized":"2023-12-04T05:40:13.326750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:16:20.208372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326517+0000","last_change":"2023-12-04T05:40:11.413970+0000","last_active":"2023-12-04T05:40:13.326517+0000","last_peered":"2023-12-04T05:40:13.326517+0000","last_clean":"2023-12-04T05:40:13.326517+0000","last_became_active":"2023-12-04T05:40:11.413680+0000","last_became_peered":"2023-12-04T05:40:11.413680+0000","last_unstale":"2023-12-04T05:40:13.326517+0000","last_undegraded":"2023-12-04T05:40:13.326517+0000","last_fullsized":"2023-12-04T05:40:13.326517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:32:53.706261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326208+0000","last_change":"2023-12-04T05:40:11.411333+0000","last_active":"2023-12-04T05:40:13.326208+0000","last_peered":"2023-12-04T05:40:13.326208+0000","last_clean":"2023-12-04T05:40:13.326208+0000","last_became_active":"2023-12-04T05:40:11.411054+0000","last_became_peered":"2023-12-04T05:40:11.411054+0000","last_unstale":"2023-12-04T05:40:13.326208+0000","last_undegraded":"2023-12-04T05:40:13.326208+0000","last_fullsized":"2023-12-04T05:40:13.326208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:21:16.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400586+0000","last_change":"2023-12-04T05:40:11.409546+0000","last_active":"2023-12-04T05:40:12.400586+0000","last_peered":"2023-12-04T05:40:12.400586+0000","last_clean":"2023-12-04T05:40:12.400586+0000","last_became_active":"2023-12-04T05:40:11.409293+0000","last_became_peered":"2023-12-04T05:40:11.409293+0000","last_unstale":"2023-12-04T05:40:12.400586+0000","last_undegraded":"2023-12-04T05:40:12.400586+0000","last_fullsized":"2023-12-04T05:40:12.400586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:11:51.087711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025563+0000","last_change":"2023-12-04T05:40:11.410085+0000","last_active":"2023-12-04T05:40:13.025563+0000","last_peered":"2023-12-04T05:40:13.025563+0000","last_clean":"2023-12-04T05:40:13.025563+0000","last_became_active":"2023-12-04T05:40:11.409814+0000","last_became_peered":"2023-12-04T05:40:11.409814+0000","last_unstale":"2023-12-04T05:40:13.025563+0000","last_undegraded":"2023-12-04T05:40:13.025563+0000","last_fullsized":"2023-12-04T05:40:13.025563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:58:01.481075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326574+0000","last_change":"2023-12-04T05:40:11.417351+0000","last_active":"2023-12-04T05:40:13.326574+0000","last_peered":"2023-12-04T05:40:13.326574+0000","last_clean":"2023-12-04T05:40:13.326574+0000","last_became_active":"2023-12-04T05:40:11.417095+0000","last_became_peered":"2023-12-04T05:40:11.417095+0000","last_unstale":"2023-12-04T05:40:13.326574+0000","last_undegraded":"2023-12-04T05:40:13.326574+0000","last_fullsized":"2023-12-04T05:40:13.326574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:10:38.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.399806+0000","last_change":"2023-12-04T05:40:11.405935+0000","last_active":"2023-12-04T05:40:12.399806+0000","last_peered":"2023-12-04T05:40:12.399806+0000","last_clean":"2023-12-04T05:40:12.399806+0000","last_became_active":"2023-12-04T05:40:11.402354+0000","last_became_peered":"2023-12-04T05:40:11.402354+0000","last_unstale":"2023-12-04T05:40:12.399806+0000","last_undegraded":"2023-12-04T05:40:12.399806+0000","last_fullsized":"2023-12-04T05:40:12.399806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:43:28.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.327088+0000","last_change":"2023-12-04T05:40:11.404034+0000","last_active":"2023-12-04T05:40:13.327088+0000","last_peered":"2023-12-04T05:40:13.327088+0000","last_clean":"2023-12-04T05:40:13.327088+0000","last_became_active":"2023-12-04T05:40:11.403714+0000","last_became_peered":"2023-12-04T05:40:11.403714+0000","last_unstale":"2023-12-04T05:40:13.327088+0000","last_undegraded":"2023-12-04T05:40:13.327088+0000","last_fullsized":"2023-12-04T05:40:13.327088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:03.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326564+0000","last_change":"2023-12-04T05:40:11.404640+0000","last_active":"2023-12-04T05:40:13.326564+0000","last_peered":"2023-12-04T05:40:13.326564+0000","last_clean":"2023-12-04T05:40:13.326564+0000","last_became_active":"2023-12-04T05:40:11.404392+0000","last_became_peered":"2023-12-04T05:40:11.404392+0000","last_unstale":"2023-12-04T05:40:13.326564+0000","last_undegraded":"2023-12-04T05:40:13.326564+0000","last_fullsized":"2023-12-04T05:40:13.326564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:44:28.365470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400644+0000","last_change":"2023-12-04T05:40:11.410366+0000","last_active":"2023-12-04T05:40:12.400644+0000","last_peered":"2023-12-04T05:40:12.400644+0000","last_clean":"2023-12-04T05:40:12.400644+0000","last_became_active":"2023-12-04T05:40:11.410095+0000","last_became_peered":"2023-12-04T05:40:11.410095+0000","last_unstale":"2023-12-04T05:40:12.400644+0000","last_undegraded":"2023-12-04T05:40:12.400644+0000","last_fullsized":"2023-12-04T05:40:12.400644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:42.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326659+0000","last_change":"2023-12-04T05:40:11.418191+0000","last_active":"2023-12-04T05:40:13.326659+0000","last_peered":"2023-12-04T05:40:13.326659+0000","last_clean":"2023-12-04T05:40:13.326659+0000","last_became_active":"2023-12-04T05:40:11.417874+0000","last_became_peered":"2023-12-04T05:40:11.417874+0000","last_unstale":"2023-12-04T05:40:13.326659+0000","last_undegraded":"2023-12-04T05:40:13.326659+0000","last_fullsized":"2023-12-04T05:40:13.326659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:44:28.365470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326748+0000","last_change":"2023-12-04T05:40:11.414358+0000","last_active":"2023-12-04T05:40:13.326748+0000","last_peered":"2023-12-04T05:40:13.326748+0000","last_clean":"2023-12-04T05:40:13.326748+0000","last_became_active":"2023-12-04T05:40:11.414063+0000","last_became_peered":"2023-12-04T05:40:11.414063+0000","last_unstale":"2023-12-04T05:40:13.326748+0000","last_undegraded":"2023-12-04T05:40:13.326748+0000","last_fullsized":"2023-12-04T05:40:13.326748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:42.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326861+0000","last_change":"2023-12-04T05:40:11.405285+0000","last_active":"2023-12-04T05:40:13.326861+0000","last_peered":"2023-12-04T05:40:13.326861+0000","last_clean":"2023-12-04T05:40:13.326861+0000","last_became_active":"2023-12-04T05:40:11.405016+0000","last_became_peered":"2023-12-04T05:40:11.405016+0000","last_unstale":"2023-12-04T05:40:13.326861+0000","last_undegraded":"2023-12-04T05:40:13.326861+0000","last_fullsized":"2023-12-04T05:40:13.326861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:12:51.236460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326338+0000","last_change":"2023-12-04T05:40:11.414341+0000","last_active":"2023-12-04T05:40:13.326338+0000","last_peered":"2023-12-04T05:40:13.326338+0000","last_clean":"2023-12-04T05:40:13.326338+0000","last_became_active":"2023-12-04T05:40:11.414054+0000","last_became_peered":"2023-12-04T05:40:11.414054+0000","last_unstale":"2023-12-04T05:40:13.326338+0000","last_undegraded":"2023-12-04T05:40:13.326338+0000","last_fullsized":"2023-12-04T05:40:13.326338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:10:38.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326387+0000","last_change":"2023-12-04T05:40:11.418259+0000","last_active":"2023-12-04T05:40:13.326387+0000","last_peered":"2023-12-04T05:40:13.326387+0000","last_clean":"2023-12-04T05:40:13.326387+0000","last_became_active":"2023-12-04T05:40:11.417969+0000","last_became_peered":"2023-12-04T05:40:11.417969+0000","last_unstale":"2023-12-04T05:40:13.326387+0000","last_undegraded":"2023-12-04T05:40:13.326387+0000","last_fullsized":"2023-12-04T05:40:13.326387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:21:54.092121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400834+0000","last_change":"2023-12-04T05:40:11.405956+0000","last_active":"2023-12-04T05:40:12.400834+0000","last_peered":"2023-12-04T05:40:12.400834+0000","last_clean":"2023-12-04T05:40:12.400834+0000","last_became_active":"2023-12-04T05:40:11.405701+0000","last_became_peered":"2023-12-04T05:40:11.405701+0000","last_unstale":"2023-12-04T05:40:12.400834+0000","last_undegraded":"2023-12-04T05:40:12.400834+0000","last_fullsized":"2023-12-04T05:40:12.400834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:21:16.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400918+0000","last_change":"2023-12-04T05:40:11.401467+0000","last_active":"2023-12-04T05:40:12.400918+0000","last_peered":"2023-12-04T05:40:12.400918+0000","last_clean":"2023-12-04T05:40:12.400918+0000","last_became_active":"2023-12-04T05:40:11.401176+0000","last_became_peered":"2023-12-04T05:40:11.401176+0000","last_unstale":"2023-12-04T05:40:12.400918+0000","last_undegraded":"2023-12-04T05:40:12.400918+0000","last_fullsized":"2023-12-04T05:40:12.400918+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:09:59.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398716+0000","last_change":"2023-12-04T05:40:11.413976+0000","last_active":"2023-12-04T05:40:12.398716+0000","last_peered":"2023-12-04T05:40:12.398716+0000","last_clean":"2023-12-04T05:40:12.398716+0000","last_became_active":"2023-12-04T05:40:11.413721+0000","last_became_peered":"2023-12-04T05:40:11.413721+0000","last_unstale":"2023-12-04T05:40:12.398716+0000","last_undegraded":"2023-12-04T05:40:12.398716+0000","last_fullsized":"2023-12-04T05:40:12.398716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:14.845449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400757+0000","last_change":"2023-12-04T05:40:11.405538+0000","last_active":"2023-12-04T05:40:12.400757+0000","last_peered":"2023-12-04T05:40:12.400757+0000","last_clean":"2023-12-04T05:40:12.400757+0000","last_became_active":"2023-12-04T05:40:11.405170+0000","last_became_peered":"2023-12-04T05:40:11.405170+0000","last_unstale":"2023-12-04T05:40:12.400757+0000","last_undegraded":"2023-12-04T05:40:12.400757+0000","last_fullsized":"2023-12-04T05:40:12.400757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:02.840080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025626+0000","last_change":"2023-12-04T05:40:11.409255+0000","last_active":"2023-12-04T05:40:13.025626+0000","last_peered":"2023-12-04T05:40:13.025626+0000","last_clean":"2023-12-04T05:40:13.025626+0000","last_became_active":"2023-12-04T05:40:11.408939+0000","last_became_peered":"2023-12-04T05:40:11.408939+0000","last_unstale":"2023-12-04T05:40:13.025626+0000","last_undegraded":"2023-12-04T05:40:13.025626+0000","last_fullsized":"2023-12-04T05:40:13.025626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:32:53.706261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326931+0000","last_change":"2023-12-04T05:40:11.407921+0000","last_active":"2023-12-04T05:40:13.326931+0000","last_peered":"2023-12-04T05:40:13.326931+0000","last_clean":"2023-12-04T05:40:13.326931+0000","last_became_active":"2023-12-04T05:40:11.407667+0000","last_became_peered":"2023-12-04T05:40:11.407667+0000","last_unstale":"2023-12-04T05:40:13.326931+0000","last_undegraded":"2023-12-04T05:40:13.326931+0000","last_fullsized":"2023-12-04T05:40:13.326931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:11:05.665982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025602+0000","last_change":"2023-12-04T05:40:11.415046+0000","last_active":"2023-12-04T05:40:13.025602+0000","last_peered":"2023-12-04T05:40:13.025602+0000","last_clean":"2023-12-04T05:40:13.025602+0000","last_became_active":"2023-12-04T05:40:11.414783+0000","last_became_peered":"2023-12-04T05:40:11.414783+0000","last_unstale":"2023-12-04T05:40:13.025602+0000","last_undegraded":"2023-12-04T05:40:13.025602+0000","last_fullsized":"2023-12-04T05:40:13.025602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:00:31.591764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.399967+0000","last_change":"2023-12-04T05:40:11.409823+0000","last_active":"2023-12-04T05:40:12.399967+0000","last_peered":"2023-12-04T05:40:12.399967+0000","last_clean":"2023-12-04T05:40:12.399967+0000","last_became_active":"2023-12-04T05:40:11.409567+0000","last_became_peered":"2023-12-04T05:40:11.409567+0000","last_unstale":"2023-12-04T05:40:12.399967+0000","last_undegraded":"2023-12-04T05:40:12.399967+0000","last_fullsized":"2023-12-04T05:40:12.399967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:58:01.481075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400784+0000","last_change":"2023-12-04T05:40:11.404475+0000","last_active":"2023-12-04T05:40:12.400784+0000","last_peered":"2023-12-04T05:40:12.400784+0000","last_clean":"2023-12-04T05:40:12.400784+0000","last_became_active":"2023-12-04T05:40:11.404163+0000","last_became_peered":"2023-12-04T05:40:11.404163+0000","last_unstale":"2023-12-04T05:40:12.400784+0000","last_undegraded":"2023-12-04T05:40:12.400784+0000","last_fullsized":"2023-12-04T05:40:12.400784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:29:23.930068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326388+0000","last_change":"2023-12-04T05:40:11.413141+0000","last_active":"2023-12-04T05:40:13.326388+0000","last_peered":"2023-12-04T05:40:13.326388+0000","last_clean":"2023-12-04T05:40:13.326388+0000","last_became_active":"2023-12-04T05:40:11.412847+0000","last_became_peered":"2023-12-04T05:40:11.412847+0000","last_unstale":"2023-12-04T05:40:13.326388+0000","last_undegraded":"2023-12-04T05:40:13.326388+0000","last_fullsized":"2023-12-04T05:40:13.326388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:43:28.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225085+0000","last_change":"2023-12-04T05:40:11.411190+0000","last_active":"2023-12-04T05:40:13.225085+0000","last_peered":"2023-12-04T05:40:13.225085+0000","last_clean":"2023-12-04T05:40:13.225085+0000","last_became_active":"2023-12-04T05:40:11.410785+0000","last_became_peered":"2023-12-04T05:40:11.410785+0000","last_unstale":"2023-12-04T05:40:13.225085+0000","last_undegraded":"2023-12-04T05:40:13.225085+0000","last_fullsized":"2023-12-04T05:40:13.225085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:43.525925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326221+0000","last_change":"2023-12-04T05:40:11.417572+0000","last_active":"2023-12-04T05:40:13.326221+0000","last_peered":"2023-12-04T05:40:13.326221+0000","last_clean":"2023-12-04T05:40:13.326221+0000","last_became_active":"2023-12-04T05:40:11.417232+0000","last_became_peered":"2023-12-04T05:40:11.417232+0000","last_unstale":"2023-12-04T05:40:13.326221+0000","last_undegraded":"2023-12-04T05:40:13.326221+0000","last_fullsized":"2023-12-04T05:40:13.326221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:09:59.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398870+0000","last_change":"2023-12-04T05:40:11.408047+0000","last_active":"2023-12-04T05:40:12.398870+0000","last_peered":"2023-12-04T05:40:12.398870+0000","last_clean":"2023-12-04T05:40:12.398870+0000","last_became_active":"2023-12-04T05:40:11.407789+0000","last_became_peered":"2023-12-04T05:40:11.407789+0000","last_unstale":"2023-12-04T05:40:12.398870+0000","last_undegraded":"2023-12-04T05:40:12.398870+0000","last_fullsized":"2023-12-04T05:40:12.398870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:11:51.087711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422526+0000","last_change":"2023-12-04T05:40:10.396846+0000","last_active":"2023-12-04T05:40:11.422526+0000","last_peered":"2023-12-04T05:40:11.422526+0000","last_clean":"2023-12-04T05:40:11.422526+0000","last_became_active":"2023-12-04T05:40:10.396580+0000","last_became_peered":"2023-12-04T05:40:10.396580+0000","last_unstale":"2023-12-04T05:40:11.422526+0000","last_undegraded":"2023-12-04T05:40:11.422526+0000","last_fullsized":"2023-12-04T05:40:11.422526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:16:48.177146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225370+0000","last_change":"2023-12-04T05:40:11.417343+0000","last_active":"2023-12-04T05:40:13.225370+0000","last_peered":"2023-12-04T05:40:13.225370+0000","last_clean":"2023-12-04T05:40:13.225370+0000","last_became_active":"2023-12-04T05:40:11.417036+0000","last_became_peered":"2023-12-04T05:40:11.417036+0000","last_unstale":"2023-12-04T05:40:13.225370+0000","last_undegraded":"2023-12-04T05:40:13.225370+0000","last_fullsized":"2023-12-04T05:40:13.225370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:38.464952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398834+0000","last_change":"2023-12-04T05:40:10.400176+0000","last_active":"2023-12-04T05:40:12.398834+0000","last_peered":"2023-12-04T05:40:12.398834+0000","last_clean":"2023-12-04T05:40:12.398834+0000","last_became_active":"2023-12-04T05:40:10.399818+0000","last_became_peered":"2023-12-04T05:40:10.399818+0000","last_unstale":"2023-12-04T05:40:12.398834+0000","last_undegraded":"2023-12-04T05:40:12.398834+0000","last_fullsized":"2023-12-04T05:40:12.398834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:42.521810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400971+0000","last_change":"2023-12-04T05:40:11.404798+0000","last_active":"2023-12-04T05:40:12.400971+0000","last_peered":"2023-12-04T05:40:12.400971+0000","last_clean":"2023-12-04T05:40:12.400971+0000","last_became_active":"2023-12-04T05:40:11.404294+0000","last_became_peered":"2023-12-04T05:40:11.404294+0000","last_unstale":"2023-12-04T05:40:12.400971+0000","last_undegraded":"2023-12-04T05:40:12.400971+0000","last_fullsized":"2023-12-04T05:40:12.400971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421369+0000","last_change":"2023-12-04T05:40:10.401657+0000","last_active":"2023-12-04T05:40:11.421369+0000","last_peered":"2023-12-04T05:40:11.421369+0000","last_clean":"2023-12-04T05:40:11.421369+0000","last_became_active":"2023-12-04T05:40:10.401385+0000","last_became_peered":"2023-12-04T05:40:10.401385+0000","last_unstale":"2023-12-04T05:40:11.421369+0000","last_undegraded":"2023-12-04T05:40:11.421369+0000","last_fullsized":"2023-12-04T05:40:11.421369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:20:48.384614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025630+0000","last_change":"2023-12-04T05:40:11.413857+0000","last_active":"2023-12-04T05:40:13.025630+0000","last_peered":"2023-12-04T05:40:13.025630+0000","last_clean":"2023-12-04T05:40:13.025630+0000","last_became_active":"2023-12-04T05:40:11.411905+0000","last_became_peered":"2023-12-04T05:40:11.411905+0000","last_unstale":"2023-12-04T05:40:13.025630+0000","last_undegraded":"2023-12-04T05:40:13.025630+0000","last_fullsized":"2023-12-04T05:40:13.025630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421180+0000","last_change":"2023-12-04T05:40:10.393071+0000","last_active":"2023-12-04T05:40:11.421180+0000","last_peered":"2023-12-04T05:40:11.421180+0000","last_clean":"2023-12-04T05:40:11.421180+0000","last_became_active":"2023-12-04T05:40:10.391756+0000","last_became_peered":"2023-12-04T05:40:10.391756+0000","last_unstale":"2023-12-04T05:40:11.421180+0000","last_undegraded":"2023-12-04T05:40:11.421180+0000","last_fullsized":"2023-12-04T05:40:11.421180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:22:57.770009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326395+0000","last_change":"2023-12-04T05:40:11.417955+0000","last_active":"2023-12-04T05:40:13.326395+0000","last_peered":"2023-12-04T05:40:13.326395+0000","last_clean":"2023-12-04T05:40:13.326395+0000","last_became_active":"2023-12-04T05:40:11.417629+0000","last_became_peered":"2023-12-04T05:40:11.417629+0000","last_unstale":"2023-12-04T05:40:13.326395+0000","last_undegraded":"2023-12-04T05:40:13.326395+0000","last_fullsized":"2023-12-04T05:40:13.326395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:58:01.481075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422068+0000","last_change":"2023-12-04T05:40:10.395884+0000","last_active":"2023-12-04T05:40:11.422068+0000","last_peered":"2023-12-04T05:40:11.422068+0000","last_clean":"2023-12-04T05:40:11.422068+0000","last_became_active":"2023-12-04T05:40:10.394514+0000","last_became_peered":"2023-12-04T05:40:10.394514+0000","last_unstale":"2023-12-04T05:40:11.422068+0000","last_undegraded":"2023-12-04T05:40:11.422068+0000","last_fullsized":"2023-12-04T05:40:11.422068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:21:53.088005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.399696+0000","last_change":"2023-12-04T05:40:11.409223+0000","last_active":"2023-12-04T05:40:12.399696+0000","last_peered":"2023-12-04T05:40:12.399696+0000","last_clean":"2023-12-04T05:40:12.399696+0000","last_became_active":"2023-12-04T05:40:11.408950+0000","last_became_peered":"2023-12-04T05:40:11.408950+0000","last_unstale":"2023-12-04T05:40:12.399696+0000","last_undegraded":"2023-12-04T05:40:12.399696+0000","last_fullsized":"2023-12-04T05:40:12.399696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421752+0000","last_change":"2023-12-04T05:40:10.395353+0000","last_active":"2023-12-04T05:40:11.421752+0000","last_peered":"2023-12-04T05:40:11.421752+0000","last_clean":"2023-12-04T05:40:11.421752+0000","last_became_active":"2023-12-04T05:40:10.395074+0000","last_became_peered":"2023-12-04T05:40:10.395074+0000","last_unstale":"2023-12-04T05:40:11.421752+0000","last_undegraded":"2023-12-04T05:40:11.421752+0000","last_fullsized":"2023-12-04T05:40:11.421752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:36:11.911650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400595+0000","last_change":"2023-12-04T05:40:11.409403+0000","last_active":"2023-12-04T05:40:12.400595+0000","last_peered":"2023-12-04T05:40:12.400595+0000","last_clean":"2023-12-04T05:40:12.400595+0000","last_became_active":"2023-12-04T05:40:11.409152+0000","last_became_peered":"2023-12-04T05:40:11.409152+0000","last_unstale":"2023-12-04T05:40:12.400595+0000","last_undegraded":"2023-12-04T05:40:12.400595+0000","last_fullsized":"2023-12-04T05:40:12.400595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:58:58.831382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.391773+0000","last_change":"2023-12-04T05:40:10.398424+0000","last_active":"2023-12-04T05:40:11.391773+0000","last_peered":"2023-12-04T05:40:11.391773+0000","last_clean":"2023-12-04T05:40:11.391773+0000","last_became_active":"2023-12-04T05:40:10.398186+0000","last_became_peered":"2023-12-04T05:40:10.398186+0000","last_unstale":"2023-12-04T05:40:11.391773+0000","last_undegraded":"2023-12-04T05:40:11.391773+0000","last_fullsized":"2023-12-04T05:40:11.391773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025189+0000","last_change":"2023-12-04T05:40:11.419281+0000","last_active":"2023-12-04T05:40:13.025189+0000","last_peered":"2023-12-04T05:40:13.025189+0000","last_clean":"2023-12-04T05:40:13.025189+0000","last_became_active":"2023-12-04T05:40:11.419019+0000","last_became_peered":"2023-12-04T05:40:11.419019+0000","last_unstale":"2023-12-04T05:40:13.025189+0000","last_undegraded":"2023-12-04T05:40:13.025189+0000","last_fullsized":"2023-12-04T05:40:13.025189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:21:16.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.394131+0000","last_change":"2023-12-04T05:40:10.393371+0000","last_active":"2023-12-04T05:40:11.394131+0000","last_peered":"2023-12-04T05:40:11.394131+0000","last_clean":"2023-12-04T05:40:11.394131+0000","last_became_active":"2023-12-04T05:40:10.393122+0000","last_became_peered":"2023-12-04T05:40:10.393122+0000","last_unstale":"2023-12-04T05:40:11.394131+0000","last_undegraded":"2023-12-04T05:40:11.394131+0000","last_fullsized":"2023-12-04T05:40:11.394131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:10:58.497714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400004+0000","last_change":"2023-12-04T05:40:11.409463+0000","last_active":"2023-12-04T05:40:12.400004+0000","last_peered":"2023-12-04T05:40:12.400004+0000","last_clean":"2023-12-04T05:40:12.400004+0000","last_became_active":"2023-12-04T05:40:11.409215+0000","last_became_peered":"2023-12-04T05:40:11.409215+0000","last_unstale":"2023-12-04T05:40:12.400004+0000","last_undegraded":"2023-12-04T05:40:12.400004+0000","last_fullsized":"2023-12-04T05:40:12.400004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:00:31.591764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.392193+0000","last_change":"2023-12-04T05:40:10.400963+0000","last_active":"2023-12-04T05:40:11.392193+0000","last_peered":"2023-12-04T05:40:11.392193+0000","last_clean":"2023-12-04T05:40:11.392193+0000","last_became_active":"2023-12-04T05:40:10.400686+0000","last_became_peered":"2023-12-04T05:40:10.400686+0000","last_unstale":"2023-12-04T05:40:11.392193+0000","last_undegraded":"2023-12-04T05:40:11.392193+0000","last_fullsized":"2023-12-04T05:40:11.392193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:58:57.827267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326870+0000","last_change":"2023-12-04T05:40:11.419905+0000","last_active":"2023-12-04T05:40:13.326870+0000","last_peered":"2023-12-04T05:40:13.326870+0000","last_clean":"2023-12-04T05:40:13.326870+0000","last_became_active":"2023-12-04T05:40:11.419620+0000","last_became_peered":"2023-12-04T05:40:11.419620+0000","last_unstale":"2023-12-04T05:40:13.326870+0000","last_undegraded":"2023-12-04T05:40:13.326870+0000","last_fullsized":"2023-12-04T05:40:13.326870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:19:19.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398915+0000","last_change":"2023-12-04T05:40:10.395994+0000","last_active":"2023-12-04T05:40:12.398915+0000","last_peered":"2023-12-04T05:40:12.398915+0000","last_clean":"2023-12-04T05:40:12.398915+0000","last_became_active":"2023-12-04T05:40:10.395726+0000","last_became_peered":"2023-12-04T05:40:10.395726+0000","last_unstale":"2023-12-04T05:40:12.398915+0000","last_undegraded":"2023-12-04T05:40:12.398915+0000","last_fullsized":"2023-12-04T05:40:12.398915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:39.600311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326610+0000","last_change":"2023-12-04T05:40:11.406971+0000","last_active":"2023-12-04T05:40:13.326610+0000","last_peered":"2023-12-04T05:40:13.326610+0000","last_clean":"2023-12-04T05:40:13.326610+0000","last_became_active":"2023-12-04T05:40:11.406713+0000","last_became_peered":"2023-12-04T05:40:11.406713+0000","last_unstale":"2023-12-04T05:40:13.326610+0000","last_undegraded":"2023-12-04T05:40:13.326610+0000","last_fullsized":"2023-12-04T05:40:13.326610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:42.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422652+0000","last_change":"2023-12-04T05:40:10.401157+0000","last_active":"2023-12-04T05:40:11.422652+0000","last_peered":"2023-12-04T05:40:11.422652+0000","last_clean":"2023-12-04T05:40:11.422652+0000","last_became_active":"2023-12-04T05:40:10.400828+0000","last_became_peered":"2023-12-04T05:40:10.400828+0000","last_unstale":"2023-12-04T05:40:11.422652+0000","last_undegraded":"2023-12-04T05:40:11.422652+0000","last_fullsized":"2023-12-04T05:40:11.422652+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:29:22.925953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400164+0000","last_change":"2023-12-04T05:40:11.409680+0000","last_active":"2023-12-04T05:40:12.400164+0000","last_peered":"2023-12-04T05:40:12.400164+0000","last_clean":"2023-12-04T05:40:12.400164+0000","last_became_active":"2023-12-04T05:40:11.409432+0000","last_became_peered":"2023-12-04T05:40:11.409432+0000","last_unstale":"2023-12-04T05:40:12.400164+0000","last_undegraded":"2023-12-04T05:40:12.400164+0000","last_fullsized":"2023-12-04T05:40:12.400164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.423675+0000","last_change":"2023-12-04T05:40:10.401551+0000","last_active":"2023-12-04T05:40:11.423675+0000","last_peered":"2023-12-04T05:40:11.423675+0000","last_clean":"2023-12-04T05:40:11.423675+0000","last_became_active":"2023-12-04T05:40:10.401316+0000","last_became_peered":"2023-12-04T05:40:10.401316+0000","last_unstale":"2023-12-04T05:40:11.423675+0000","last_undegraded":"2023-12-04T05:40:11.423675+0000","last_fullsized":"2023-12-04T05:40:11.423675+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:37.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025188+0000","last_change":"2023-12-04T05:40:11.419368+0000","last_active":"2023-12-04T05:40:13.025188+0000","last_peered":"2023-12-04T05:40:13.025188+0000","last_clean":"2023-12-04T05:40:13.025188+0000","last_became_active":"2023-12-04T05:40:11.419094+0000","last_became_peered":"2023-12-04T05:40:11.419094+0000","last_unstale":"2023-12-04T05:40:13.025188+0000","last_undegraded":"2023-12-04T05:40:13.025188+0000","last_fullsized":"2023-12-04T05:40:13.025188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:09:59.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422014+0000","last_change":"2023-12-04T05:40:10.390589+0000","last_active":"2023-12-04T05:40:11.422014+0000","last_peered":"2023-12-04T05:40:11.422014+0000","last_clean":"2023-12-04T05:40:11.422014+0000","last_became_active":"2023-12-04T05:40:10.390289+0000","last_became_peered":"2023-12-04T05:40:10.390289+0000","last_unstale":"2023-12-04T05:40:11.422014+0000","last_undegraded":"2023-12-04T05:40:11.422014+0000","last_fullsized":"2023-12-04T05:40:11.422014+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326635+0000","last_change":"2023-12-04T05:40:11.403109+0000","last_active":"2023-12-04T05:40:13.326635+0000","last_peered":"2023-12-04T05:40:13.326635+0000","last_clean":"2023-12-04T05:40:13.326635+0000","last_became_active":"2023-12-04T05:40:11.402842+0000","last_became_peered":"2023-12-04T05:40:11.402842+0000","last_unstale":"2023-12-04T05:40:13.326635+0000","last_undegraded":"2023-12-04T05:40:13.326635+0000","last_fullsized":"2023-12-04T05:40:13.326635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:02.840080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.402390+0000","last_change":"2023-12-04T05:40:10.397172+0000","last_active":"2023-12-04T05:40:11.402390+0000","last_peered":"2023-12-04T05:40:11.402390+0000","last_clean":"2023-12-04T05:40:11.402390+0000","last_became_active":"2023-12-04T05:40:10.396752+0000","last_became_peered":"2023-12-04T05:40:10.396752+0000","last_unstale":"2023-12-04T05:40:11.402390+0000","last_undegraded":"2023-12-04T05:40:11.402390+0000","last_fullsized":"2023-12-04T05:40:11.402390+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:39.600311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326477+0000","last_change":"2023-12-04T05:40:11.417904+0000","last_active":"2023-12-04T05:40:13.326477+0000","last_peered":"2023-12-04T05:40:13.326477+0000","last_clean":"2023-12-04T05:40:13.326477+0000","last_became_active":"2023-12-04T05:40:11.417618+0000","last_became_peered":"2023-12-04T05:40:11.417618+0000","last_unstale":"2023-12-04T05:40:13.326477+0000","last_undegraded":"2023-12-04T05:40:13.326477+0000","last_fullsized":"2023-12-04T05:40:13.326477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:00:31.591764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.391852+0000","last_change":"2023-12-04T05:40:10.399192+0000","last_active":"2023-12-04T05:40:11.391852+0000","last_peered":"2023-12-04T05:40:11.391852+0000","last_clean":"2023-12-04T05:40:11.391852+0000","last_became_active":"2023-12-04T05:40:10.398915+0000","last_became_peered":"2023-12-04T05:40:10.398915+0000","last_unstale":"2023-12-04T05:40:11.391852+0000","last_undegraded":"2023-12-04T05:40:11.391852+0000","last_fullsized":"2023-12-04T05:40:11.391852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:42.521810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225205+0000","last_change":"2023-12-04T05:40:11.417361+0000","last_active":"2023-12-04T05:40:13.225205+0000","last_peered":"2023-12-04T05:40:13.225205+0000","last_clean":"2023-12-04T05:40:13.225205+0000","last_became_active":"2023-12-04T05:40:11.417067+0000","last_became_peered":"2023-12-04T05:40:11.417067+0000","last_unstale":"2023-12-04T05:40:13.225205+0000","last_undegraded":"2023-12-04T05:40:13.225205+0000","last_fullsized":"2023-12-04T05:40:13.225205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:39:57.822988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421998+0000","last_change":"2023-12-04T05:40:10.401519+0000","last_active":"2023-12-04T05:40:11.421998+0000","last_peered":"2023-12-04T05:40:11.421998+0000","last_clean":"2023-12-04T05:40:11.421998+0000","last_became_active":"2023-12-04T05:40:10.401274+0000","last_became_peered":"2023-12-04T05:40:10.401274+0000","last_unstale":"2023-12-04T05:40:11.421998+0000","last_undegraded":"2023-12-04T05:40:11.421998+0000","last_fullsized":"2023-12-04T05:40:11.421998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400161+0000","last_change":"2023-12-04T05:40:11.410967+0000","last_active":"2023-12-04T05:40:12.400161+0000","last_peered":"2023-12-04T05:40:12.400161+0000","last_clean":"2023-12-04T05:40:12.400161+0000","last_became_active":"2023-12-04T05:40:11.410547+0000","last_became_peered":"2023-12-04T05:40:11.410547+0000","last_unstale":"2023-12-04T05:40:12.400161+0000","last_undegraded":"2023-12-04T05:40:12.400161+0000","last_fullsized":"2023-12-04T05:40:12.400161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:38.464952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421450+0000","last_change":"2023-12-04T05:40:10.397087+0000","last_active":"2023-12-04T05:40:11.421450+0000","last_peered":"2023-12-04T05:40:11.421450+0000","last_clean":"2023-12-04T05:40:11.421450+0000","last_became_active":"2023-12-04T05:40:10.396853+0000","last_became_peered":"2023-12-04T05:40:10.396853+0000","last_unstale":"2023-12-04T05:40:11.421450+0000","last_undegraded":"2023-12-04T05:40:11.421450+0000","last_fullsized":"2023-12-04T05:40:11.421450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:37.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400051+0000","last_change":"2023-12-04T05:40:11.403287+0000","last_active":"2023-12-04T05:40:12.400051+0000","last_peered":"2023-12-04T05:40:12.400051+0000","last_clean":"2023-12-04T05:40:12.400051+0000","last_became_active":"2023-12-04T05:40:11.403045+0000","last_became_peered":"2023-12-04T05:40:11.403045+0000","last_unstale":"2023-12-04T05:40:12.400051+0000","last_undegraded":"2023-12-04T05:40:12.400051+0000","last_fullsized":"2023-12-04T05:40:12.400051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:32:53.706261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421653+0000","last_change":"2023-12-04T05:40:10.391094+0000","last_active":"2023-12-04T05:40:11.421653+0000","last_peered":"2023-12-04T05:40:11.421653+0000","last_clean":"2023-12-04T05:40:11.421653+0000","last_became_active":"2023-12-04T05:40:10.390763+0000","last_became_peered":"2023-12-04T05:40:10.390763+0000","last_unstale":"2023-12-04T05:40:11.421653+0000","last_undegraded":"2023-12-04T05:40:11.421653+0000","last_fullsized":"2023-12-04T05:40:11.421653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:13.841334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025362+0000","last_change":"2023-12-04T05:40:11.419232+0000","last_active":"2023-12-04T05:40:13.025362+0000","last_peered":"2023-12-04T05:40:13.025362+0000","last_clean":"2023-12-04T05:40:13.025362+0000","last_became_active":"2023-12-04T05:40:11.418953+0000","last_became_peered":"2023-12-04T05:40:11.418953+0000","last_unstale":"2023-12-04T05:40:13.025362+0000","last_undegraded":"2023-12-04T05:40:13.025362+0000","last_fullsized":"2023-12-04T05:40:13.025362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:43:28.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421528+0000","last_change":"2023-12-04T05:40:10.395628+0000","last_active":"2023-12-04T05:40:11.421528+0000","last_peered":"2023-12-04T05:40:11.421528+0000","last_clean":"2023-12-04T05:40:11.421528+0000","last_became_active":"2023-12-04T05:40:10.395276+0000","last_became_peered":"2023-12-04T05:40:10.395276+0000","last_unstale":"2023-12-04T05:40:11.421528+0000","last_undegraded":"2023-12-04T05:40:11.421528+0000","last_fullsized":"2023-12-04T05:40:11.421528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326291+0000","last_change":"2023-12-04T05:40:11.408455+0000","last_active":"2023-12-04T05:40:13.326291+0000","last_peered":"2023-12-04T05:40:13.326291+0000","last_clean":"2023-12-04T05:40:13.326291+0000","last_became_active":"2023-12-04T05:40:11.408109+0000","last_became_peered":"2023-12-04T05:40:11.408109+0000","last_unstale":"2023-12-04T05:40:13.326291+0000","last_undegraded":"2023-12-04T05:40:13.326291+0000","last_fullsized":"2023-12-04T05:40:13.326291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:36:12.915765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.395872+0000","last_change":"2023-12-04T05:40:10.393607+0000","last_active":"2023-12-04T05:40:11.395872+0000","last_peered":"2023-12-04T05:40:11.395872+0000","last_clean":"2023-12-04T05:40:11.395872+0000","last_became_active":"2023-12-04T05:40:10.393352+0000","last_became_peered":"2023-12-04T05:40:10.393352+0000","last_unstale":"2023-12-04T05:40:11.395872+0000","last_undegraded":"2023-12-04T05:40:11.395872+0000","last_fullsized":"2023-12-04T05:40:11.395872+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:39.600311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326197+0000","last_change":"2023-12-04T05:40:11.413442+0000","last_active":"2023-12-04T05:40:13.326197+0000","last_peered":"2023-12-04T05:40:13.326197+0000","last_clean":"2023-12-04T05:40:13.326197+0000","last_became_active":"2023-12-04T05:40:11.413182+0000","last_became_peered":"2023-12-04T05:40:11.413182+0000","last_unstale":"2023-12-04T05:40:13.326197+0000","last_undegraded":"2023-12-04T05:40:13.326197+0000","last_fullsized":"2023-12-04T05:40:13.326197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:43:10.132992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398884+0000","last_change":"2023-12-04T05:40:10.396044+0000","last_active":"2023-12-04T05:40:12.398884+0000","last_peered":"2023-12-04T05:40:12.398884+0000","last_clean":"2023-12-04T05:40:12.398884+0000","last_became_active":"2023-12-04T05:40:10.395788+0000","last_became_peered":"2023-12-04T05:40:10.395788+0000","last_unstale":"2023-12-04T05:40:12.398884+0000","last_undegraded":"2023-12-04T05:40:12.398884+0000","last_fullsized":"2023-12-04T05:40:12.398884+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.327111+0000","last_change":"2023-12-04T05:40:11.412904+0000","last_active":"2023-12-04T05:40:13.327111+0000","last_peered":"2023-12-04T05:40:13.327111+0000","last_clean":"2023-12-04T05:40:13.327111+0000","last_became_active":"2023-12-04T05:40:11.412643+0000","last_became_peered":"2023-12-04T05:40:11.412643+0000","last_unstale":"2023-12-04T05:40:13.327111+0000","last_undegraded":"2023-12-04T05:40:13.327111+0000","last_fullsized":"2023-12-04T05:40:13.327111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:43.525925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.403723+0000","last_change":"2023-12-04T05:40:10.397551+0000","last_active":"2023-12-04T05:40:11.403723+0000","last_peered":"2023-12-04T05:40:11.403723+0000","last_clean":"2023-12-04T05:40:11.403723+0000","last_became_active":"2023-12-04T05:40:10.397198+0000","last_became_peered":"2023-12-04T05:40:10.397198+0000","last_unstale":"2023-12-04T05:40:11.403723+0000","last_undegraded":"2023-12-04T05:40:11.403723+0000","last_fullsized":"2023-12-04T05:40:11.403723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326262+0000","last_change":"2023-12-04T05:40:11.410058+0000","last_active":"2023-12-04T05:40:13.326262+0000","last_peered":"2023-12-04T05:40:13.326262+0000","last_clean":"2023-12-04T05:40:13.326262+0000","last_became_active":"2023-12-04T05:40:11.409790+0000","last_became_peered":"2023-12-04T05:40:11.409790+0000","last_unstale":"2023-12-04T05:40:13.326262+0000","last_undegraded":"2023-12-04T05:40:13.326262+0000","last_fullsized":"2023-12-04T05:40:13.326262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:03:44.891656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.393250+0000","last_change":"2023-12-04T05:40:10.392389+0000","last_active":"2023-12-04T05:40:11.393250+0000","last_peered":"2023-12-04T05:40:11.393250+0000","last_clean":"2023-12-04T05:40:11.393250+0000","last_became_active":"2023-12-04T05:40:10.390893+0000","last_became_peered":"2023-12-04T05:40:10.390893+0000","last_unstale":"2023-12-04T05:40:11.393250+0000","last_undegraded":"2023-12-04T05:40:11.393250+0000","last_fullsized":"2023-12-04T05:40:11.393250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:12:01.216530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326307+0000","last_change":"2023-12-04T05:40:11.413170+0000","last_active":"2023-12-04T05:40:13.326307+0000","last_peered":"2023-12-04T05:40:13.326307+0000","last_clean":"2023-12-04T05:40:13.326307+0000","last_became_active":"2023-12-04T05:40:11.412875+0000","last_became_peered":"2023-12-04T05:40:11.412875+0000","last_unstale":"2023-12-04T05:40:13.326307+0000","last_undegraded":"2023-12-04T05:40:13.326307+0000","last_fullsized":"2023-12-04T05:40:13.326307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.393035+0000","last_change":"2023-12-04T05:40:10.400242+0000","last_active":"2023-12-04T05:40:11.393035+0000","last_peered":"2023-12-04T05:40:11.393035+0000","last_clean":"2023-12-04T05:40:11.393035+0000","last_became_active":"2023-12-04T05:40:10.399939+0000","last_became_peered":"2023-12-04T05:40:10.399939+0000","last_unstale":"2023-12-04T05:40:11.393035+0000","last_undegraded":"2023-12-04T05:40:11.393035+0000","last_fullsized":"2023-12-04T05:40:11.393035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326263+0000","last_change":"2023-12-04T05:40:11.417608+0000","last_active":"2023-12-04T05:40:13.326263+0000","last_peered":"2023-12-04T05:40:13.326263+0000","last_clean":"2023-12-04T05:40:13.326263+0000","last_became_active":"2023-12-04T05:40:11.417350+0000","last_became_peered":"2023-12-04T05:40:11.417350+0000","last_unstale":"2023-12-04T05:40:13.326263+0000","last_undegraded":"2023-12-04T05:40:13.326263+0000","last_fullsized":"2023-12-04T05:40:13.326263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:16:49.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422193+0000","last_change":"2023-12-04T05:40:10.396052+0000","last_active":"2023-12-04T05:40:11.422193+0000","last_peered":"2023-12-04T05:40:11.422193+0000","last_clean":"2023-12-04T05:40:11.422193+0000","last_became_active":"2023-12-04T05:40:10.395808+0000","last_became_peered":"2023-12-04T05:40:10.395808+0000","last_unstale":"2023-12-04T05:40:11.422193+0000","last_undegraded":"2023-12-04T05:40:11.422193+0000","last_fullsized":"2023-12-04T05:40:11.422193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:11:50.083596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400223+0000","last_change":"2023-12-04T05:40:11.409215+0000","last_active":"2023-12-04T05:40:12.400223+0000","last_peered":"2023-12-04T05:40:12.400223+0000","last_clean":"2023-12-04T05:40:12.400223+0000","last_became_active":"2023-12-04T05:40:11.408937+0000","last_became_peered":"2023-12-04T05:40:11.408937+0000","last_unstale":"2023-12-04T05:40:12.400223+0000","last_undegraded":"2023-12-04T05:40:12.400223+0000","last_fullsized":"2023-12-04T05:40:12.400223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:10:38.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398949+0000","last_change":"2023-12-04T05:40:10.396822+0000","last_active":"2023-12-04T05:40:12.398949+0000","last_peered":"2023-12-04T05:40:12.398949+0000","last_clean":"2023-12-04T05:40:12.398949+0000","last_became_active":"2023-12-04T05:40:10.396542+0000","last_became_peered":"2023-12-04T05:40:10.396542+0000","last_unstale":"2023-12-04T05:40:12.398949+0000","last_undegraded":"2023-12-04T05:40:12.398949+0000","last_fullsized":"2023-12-04T05:40:12.398949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:39:56.818872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398792+0000","last_change":"2023-12-04T05:40:11.415224+0000","last_active":"2023-12-04T05:40:12.398792+0000","last_peered":"2023-12-04T05:40:12.398792+0000","last_clean":"2023-12-04T05:40:12.398792+0000","last_became_active":"2023-12-04T05:40:11.414961+0000","last_became_peered":"2023-12-04T05:40:11.414961+0000","last_unstale":"2023-12-04T05:40:12.398792+0000","last_undegraded":"2023-12-04T05:40:12.398792+0000","last_fullsized":"2023-12-04T05:40:12.398792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:38.464952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.408480+0000","last_change":"2023-12-04T05:40:10.394243+0000","last_active":"2023-12-04T05:40:11.408480+0000","last_peered":"2023-12-04T05:40:11.408480+0000","last_clean":"2023-12-04T05:40:11.408480+0000","last_became_active":"2023-12-04T05:40:10.393531+0000","last_became_peered":"2023-12-04T05:40:10.393531+0000","last_unstale":"2023-12-04T05:40:11.408480+0000","last_undegraded":"2023-12-04T05:40:11.408480+0000","last_fullsized":"2023-12-04T05:40:11.408480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:20:48.384614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225227+0000","last_change":"2023-12-04T05:40:11.412378+0000","last_active":"2023-12-04T05:40:13.225227+0000","last_peered":"2023-12-04T05:40:13.225227+0000","last_clean":"2023-12-04T05:40:13.225227+0000","last_became_active":"2023-12-04T05:40:11.412086+0000","last_became_peered":"2023-12-04T05:40:11.412086+0000","last_unstale":"2023-12-04T05:40:13.225227+0000","last_undegraded":"2023-12-04T05:40:13.225227+0000","last_fullsized":"2023-12-04T05:40:13.225227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:40.604427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398963+0000","last_change":"2023-12-04T05:40:10.397176+0000","last_active":"2023-12-04T05:40:12.398963+0000","last_peered":"2023-12-04T05:40:12.398963+0000","last_clean":"2023-12-04T05:40:12.398963+0000","last_became_active":"2023-12-04T05:40:10.396863+0000","last_became_peered":"2023-12-04T05:40:10.396863+0000","last_unstale":"2023-12-04T05:40:12.398963+0000","last_undegraded":"2023-12-04T05:40:12.398963+0000","last_fullsized":"2023-12-04T05:40:12.398963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398891+0000","last_change":"2023-12-04T05:40:11.411082+0000","last_active":"2023-12-04T05:40:12.398891+0000","last_peered":"2023-12-04T05:40:12.398891+0000","last_clean":"2023-12-04T05:40:12.398891+0000","last_became_active":"2023-12-04T05:40:11.410549+0000","last_became_peered":"2023-12-04T05:40:11.410549+0000","last_unstale":"2023-12-04T05:40:12.398891+0000","last_undegraded":"2023-12-04T05:40:12.398891+0000","last_fullsized":"2023-12-04T05:40:12.398891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:58:58.831382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.402499+0000","last_change":"2023-12-04T05:40:10.397000+0000","last_active":"2023-12-04T05:40:11.402499+0000","last_peered":"2023-12-04T05:40:11.402499+0000","last_clean":"2023-12-04T05:40:11.402499+0000","last_became_active":"2023-12-04T05:40:10.396440+0000","last_became_peered":"2023-12-04T05:40:10.396440+0000","last_unstale":"2023-12-04T05:40:11.402499+0000","last_undegraded":"2023-12-04T05:40:11.402499+0000","last_fullsized":"2023-12-04T05:40:11.402499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:13.841334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326100+0000","last_change":"2023-12-04T05:40:11.405888+0000","last_active":"2023-12-04T05:40:13.326100+0000","last_peered":"2023-12-04T05:40:13.326100+0000","last_clean":"2023-12-04T05:40:13.326100+0000","last_became_active":"2023-12-04T05:40:11.405634+0000","last_became_peered":"2023-12-04T05:40:11.405634+0000","last_unstale":"2023-12-04T05:40:13.326100+0000","last_undegraded":"2023-12-04T05:40:13.326100+0000","last_fullsized":"2023-12-04T05:40:13.326100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.394201+0000","last_change":"2023-12-04T05:40:10.392063+0000","last_active":"2023-12-04T05:40:11.394201+0000","last_peered":"2023-12-04T05:40:11.394201+0000","last_clean":"2023-12-04T05:40:11.394201+0000","last_became_active":"2023-12-04T05:40:10.391681+0000","last_became_peered":"2023-12-04T05:40:10.391681+0000","last_unstale":"2023-12-04T05:40:11.394201+0000","last_undegraded":"2023-12-04T05:40:11.394201+0000","last_fullsized":"2023-12-04T05:40:11.394201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:22:57.770009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400767+0000","last_change":"2023-12-04T05:40:11.403813+0000","last_active":"2023-12-04T05:40:12.400767+0000","last_peered":"2023-12-04T05:40:12.400767+0000","last_clean":"2023-12-04T05:40:12.400767+0000","last_became_active":"2023-12-04T05:40:11.402867+0000","last_became_peered":"2023-12-04T05:40:11.402867+0000","last_unstale":"2023-12-04T05:40:12.400767+0000","last_undegraded":"2023-12-04T05:40:12.400767+0000","last_fullsized":"2023-12-04T05:40:12.400767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:20:49.388730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.391893+0000","last_change":"2023-12-04T05:40:10.400004+0000","last_active":"2023-12-04T05:40:11.391893+0000","last_peered":"2023-12-04T05:40:11.391893+0000","last_clean":"2023-12-04T05:40:11.391893+0000","last_became_active":"2023-12-04T05:40:10.399698+0000","last_became_peered":"2023-12-04T05:40:10.399698+0000","last_unstale":"2023-12-04T05:40:11.391893+0000","last_undegraded":"2023-12-04T05:40:11.391893+0000","last_fullsized":"2023-12-04T05:40:11.391893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:39:56.818872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326513+0000","last_change":"2023-12-04T05:40:11.412217+0000","last_active":"2023-12-04T05:40:13.326513+0000","last_peered":"2023-12-04T05:40:13.326513+0000","last_clean":"2023-12-04T05:40:13.326513+0000","last_became_active":"2023-12-04T05:40:11.411959+0000","last_became_peered":"2023-12-04T05:40:11.411959+0000","last_unstale":"2023-12-04T05:40:13.326513+0000","last_undegraded":"2023-12-04T05:40:13.326513+0000","last_fullsized":"2023-12-04T05:40:13.326513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.403135+0000","last_change":"2023-12-04T05:40:10.398575+0000","last_active":"2023-12-04T05:40:11.403135+0000","last_peered":"2023-12-04T05:40:11.403135+0000","last_clean":"2023-12-04T05:40:11.403135+0000","last_became_active":"2023-12-04T05:40:10.398318+0000","last_became_peered":"2023-12-04T05:40:10.398318+0000","last_unstale":"2023-12-04T05:40:11.403135+0000","last_undegraded":"2023-12-04T05:40:11.403135+0000","last_fullsized":"2023-12-04T05:40:11.403135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:36:11.911650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.405851+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:40:11.405851+0000","last_peered":"2023-12-04T05:40:11.405851+0000","last_clean":"2023-12-04T05:40:11.405851+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:40:11.405851+0000","last_undegraded":"2023-12-04T05:40:11.405851+0000","last_fullsized":"2023-12-04T05:40:11.405851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.435}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.5}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44500000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49199999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69699999999999995}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.748}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72799999999999998}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.627},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.51900000000000002}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.372},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.311}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48599999999999999}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.498}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51400000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.26700000000000002}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.314}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33000000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.498},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67400000000000004}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.35499999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.36899999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.311},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.40000000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.871}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:40:19.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:40:19.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.840+0000 7f33b3290640 1 Processor -- start 2023-12-04T05:40:19.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.841+0000 7f33b3290640 1 -- start start 2023-12-04T05:40:19.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.841+0000 7f33b3290640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33ac13cee0 0x7f33ac06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:19.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.841+0000 7f33b3290640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33ac15aba0 0x7f33ac06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:19.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.841+0000 7f33b3290640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac0aea20 con 0x7f33ac140200 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.841+0000 7f33b3290640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac13d2c0 con 0x7f33ac15aba0 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.841+0000 7f33b3290640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac15b3a0 con 0x7f33ac13cee0 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33ac15aba0 0x7f33ac06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33ac15aba0 0x7f33ac06c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42162/0 (socket says 172.21.15.111:42162) 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 -- 172.21.15.111:0/129166194 learned_addr learned my addr 172.21.15.111:0/129166194 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b102c640 1 --2- 172.21.15.111:0/129166194 >> v2:172.21.15.192:3300/0 conn(0x7f33ac13cee0 0x7f33ac06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 -- 172.21.15.111:0/129166194 >> v2:172.21.15.192:3300/0 conn(0x7f33ac13cee0 msgr2=0x7f33ac06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 --2- 172.21.15.111:0/129166194 >> v2:172.21.15.192:3300/0 conn(0x7f33ac13cee0 0x7f33ac06c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:19.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 -- 172.21.15.111:0/129166194 >> v1:172.21.15.192:6789/0 conn(0x7f33ac140200 legacy=0x7f33ac15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.842+0000 7f33b182d640 1 -- 172.21.15.111:0/129166194 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac15b0c0 con 0x7f33ac15aba0 2023-12-04T05:40:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b182d640 1 --2- 172.21.15.111:0/129166194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac15aba0 0x7f33ac06c790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0b2280 tx=0x7f33a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cebfa843d68a788 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33a3fff640 1 -- 172.21.15.111:0/129166194 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a802c9f0 con 0x7f33ac15aba0 2023-12-04T05:40:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33a3fff640 1 -- 172.21.15.111:0/129166194 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a802cb50 con 0x7f33ac15aba0 2023-12-04T05:40:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33a3fff640 1 -- 172.21.15.111:0/129166194 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80343f0 con 0x7f33ac15aba0 2023-12-04T05:40:19.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b3290640 1 -- 172.21.15.111:0/129166194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac15aba0 msgr2=0x7f33ac06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b3290640 1 --2- 172.21.15.111:0/129166194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac15aba0 0x7f33ac06c790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0b2280 tx=0x7f33a802be50 comp rx=0 tx=0).stop 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b3290640 1 -- 172.21.15.111:0/129166194 shutdown_connections 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b3290640 1 --2- 172.21.15.111:0/129166194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac15aba0 0x7f33ac06c790 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b3290640 1 --2- 172.21.15.111:0/129166194 >> v2:172.21.15.192:3300/0 conn(0x7f33ac13cee0 0x7f33ac06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.843+0000 7f33b3290640 1 -- 172.21.15.111:0/129166194 >> 172.21.15.111:0/129166194 conn(0x7f33ac06a4c0 msgr2=0x7f33ac06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.844+0000 7f33b3290640 1 -- 172.21.15.111:0/129166194 shutdown_connections 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.844+0000 7f33b3290640 1 -- 172.21.15.111:0/129166194 wait complete. 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.845+0000 7f33b3290640 1 Processor -- start 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.845+0000 7f33b3290640 1 -- start start 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 0x7f33ac1f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33ac140200 0x7f33ac1316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33ac15aba0 0x7f33ac0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1543e0 con 0x7f33ac13cee0 2023-12-04T05:40:19.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1f4e60 con 0x7f33ac15aba0 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1f5140 con 0x7f33ac140200 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 0x7f33ac1f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 0x7f33ac1f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42172/0 (socket says 172.21.15.111:42172) 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 -- 172.21.15.111:0/3889537383 learned_addr learned my addr 172.21.15.111:0/3889537383 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b082b640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33ac140200 0x7f33ac1316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b182d640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33ac15aba0 0x7f33ac0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 -- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33ac140200 msgr2=0x7f33ac1316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33ac140200 0x7f33ac1316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 -- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33ac15aba0 msgr2=0x7f33ac0719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33ac15aba0 0x7f33ac0719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 -- 172.21.15.111:0/3889537383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009b30 con 0x7f33ac13cee0 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b182d640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33ac15aba0 0x7f33ac0719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b102c640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 0x7f33ac1f1880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c7f0 tx=0x7f33a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:19.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4004220 con 0x7f33ac13cee0 2023-12-04T05:40:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a40043a0 con 0x7f33ac13cee0 2023-12-04T05:40:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4007860 con 0x7f33ac13cee0 2023-12-04T05:40:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.846+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac1f0da0 con 0x7f33ac13cee0 2023-12-04T05:40:19.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.847+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac1f1360 con 0x7f33ac13cee0 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.848+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac1f1550 con 0x7f33ac13cee0 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.848+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f33a4007bd0 con 0x7f33ac13cee0 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.848+0000 7f33a1ffb640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f339006b2d0 0x7f339006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.848+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33a4089b90 con 0x7f33ac13cee0 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.848+0000 7f33b082b640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f339006b2d0 0x7f339006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.849+0000 7f33b082b640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f339006b2d0 0x7f339006d790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f33ac158470 tx=0x7f339c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:19.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:19.853+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33a4054a00 con 0x7f33ac13cee0 2023-12-04T05:40:20.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.014+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33ac13fce0 con 0x7f339006b2d0 2023-12-04T05:40:20.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.022+0000 7f33a1ffb640 1 -- 172.21.15.111:0/3889537383 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290025 (secure 0 0 0) 0x7f339c0053f0 con 0x7f339006b2d0 2023-12-04T05:40:20.027 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:20.029 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f339006b2d0 msgr2=0x7f339006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f339006b2d0 0x7f339006d790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f33ac158470 tx=0x7f339c009040 comp rx=0 tx=0).stop 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 msgr2=0x7f33ac1f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 0x7f33ac1f1880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c7f0 tx=0x7f33a400ccb0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 shutdown_connections 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f339006b2d0 0x7f339006d790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33ac15aba0 0x7f33ac0719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33ac140200 0x7f33ac1316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 --2- 172.21.15.111:0/3889537383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33ac13cee0 0x7f33ac1f1880 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 >> 172.21.15.111:0/3889537383 conn(0x7f33ac06a4c0 msgr2=0x7f33ac15a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 shutdown_connections 2023-12-04T05:40:20.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.025+0000 7f33b3290640 1 -- 172.21.15.111:0/3889537383 wait complete. 2023-12-04T05:40:20.037 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-12-04T05:40:18.484366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001393"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225242+0000","last_change":"2023-12-04T05:40:11.422578+0000","last_active":"2023-12-04T05:40:13.225242+0000","last_peered":"2023-12-04T05:40:13.225242+0000","last_clean":"2023-12-04T05:40:13.225242+0000","last_became_active":"2023-12-04T05:40:11.422151+0000","last_became_peered":"2023-12-04T05:40:11.422151+0000","last_unstale":"2023-12-04T05:40:13.225242+0000","last_undegraded":"2023-12-04T05:40:13.225242+0000","last_fullsized":"2023-12-04T05:40:13.225242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:14.845449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326842+0000","last_change":"2023-12-04T05:40:11.407146+0000","last_active":"2023-12-04T05:40:13.326842+0000","last_peered":"2023-12-04T05:40:13.326842+0000","last_clean":"2023-12-04T05:40:13.326842+0000","last_became_active":"2023-12-04T05:40:11.406358+0000","last_became_peered":"2023-12-04T05:40:11.406358+0000","last_unstale":"2023-12-04T05:40:13.326842+0000","last_undegraded":"2023-12-04T05:40:13.326842+0000","last_fullsized":"2023-12-04T05:40:13.326842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:47:31.152090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400287+0000","last_change":"2023-12-04T05:40:11.407620+0000","last_active":"2023-12-04T05:40:12.400287+0000","last_peered":"2023-12-04T05:40:12.400287+0000","last_clean":"2023-12-04T05:40:12.400287+0000","last_became_active":"2023-12-04T05:40:11.407370+0000","last_became_peered":"2023-12-04T05:40:11.407370+0000","last_unstale":"2023-12-04T05:40:12.400287+0000","last_undegraded":"2023-12-04T05:40:12.400287+0000","last_fullsized":"2023-12-04T05:40:12.400287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:44:28.365470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025333+0000","last_change":"2023-12-04T05:40:11.415373+0000","last_active":"2023-12-04T05:40:13.025333+0000","last_peered":"2023-12-04T05:40:13.025333+0000","last_clean":"2023-12-04T05:40:13.025333+0000","last_became_active":"2023-12-04T05:40:11.415117+0000","last_became_peered":"2023-12-04T05:40:11.415117+0000","last_unstale":"2023-12-04T05:40:13.025333+0000","last_undegraded":"2023-12-04T05:40:13.025333+0000","last_fullsized":"2023-12-04T05:40:13.025333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.327031+0000","last_change":"2023-12-04T05:40:11.407975+0000","last_active":"2023-12-04T05:40:13.327031+0000","last_peered":"2023-12-04T05:40:13.327031+0000","last_clean":"2023-12-04T05:40:13.327031+0000","last_became_active":"2023-12-04T05:40:11.407721+0000","last_became_peered":"2023-12-04T05:40:11.407721+0000","last_unstale":"2023-12-04T05:40:13.327031+0000","last_undegraded":"2023-12-04T05:40:13.327031+0000","last_fullsized":"2023-12-04T05:40:13.327031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:52.381604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326750+0000","last_change":"2023-12-04T05:40:11.396951+0000","last_active":"2023-12-04T05:40:13.326750+0000","last_peered":"2023-12-04T05:40:13.326750+0000","last_clean":"2023-12-04T05:40:13.326750+0000","last_became_active":"2023-12-04T05:40:11.396700+0000","last_became_peered":"2023-12-04T05:40:11.396700+0000","last_unstale":"2023-12-04T05:40:13.326750+0000","last_undegraded":"2023-12-04T05:40:13.326750+0000","last_fullsized":"2023-12-04T05:40:13.326750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:16:20.208372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326517+0000","last_change":"2023-12-04T05:40:11.413970+0000","last_active":"2023-12-04T05:40:13.326517+0000","last_peered":"2023-12-04T05:40:13.326517+0000","last_clean":"2023-12-04T05:40:13.326517+0000","last_became_active":"2023-12-04T05:40:11.413680+0000","last_became_peered":"2023-12-04T05:40:11.413680+0000","last_unstale":"2023-12-04T05:40:13.326517+0000","last_undegraded":"2023-12-04T05:40:13.326517+0000","last_fullsized":"2023-12-04T05:40:13.326517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:32:53.706261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326208+0000","last_change":"2023-12-04T05:40:11.411333+0000","last_active":"2023-12-04T05:40:13.326208+0000","last_peered":"2023-12-04T05:40:13.326208+0000","last_clean":"2023-12-04T05:40:13.326208+0000","last_became_active":"2023-12-04T05:40:11.411054+0000","last_became_peered":"2023-12-04T05:40:11.411054+0000","last_unstale":"2023-12-04T05:40:13.326208+0000","last_undegraded":"2023-12-04T05:40:13.326208+0000","last_fullsized":"2023-12-04T05:40:13.326208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:21:16.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400586+0000","last_change":"2023-12-04T05:40:11.409546+0000","last_active":"2023-12-04T05:40:12.400586+0000","last_peered":"2023-12-04T05:40:12.400586+0000","last_clean":"2023-12-04T05:40:12.400586+0000","last_became_active":"2023-12-04T05:40:11.409293+0000","last_became_peered":"2023-12-04T05:40:11.409293+0000","last_unstale":"2023-12-04T05:40:12.400586+0000","last_undegraded":"2023-12-04T05:40:12.400586+0000","last_fullsized":"2023-12-04T05:40:12.400586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:11:51.087711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025563+0000","last_change":"2023-12-04T05:40:11.410085+0000","last_active":"2023-12-04T05:40:13.025563+0000","last_peered":"2023-12-04T05:40:13.025563+0000","last_clean":"2023-12-04T05:40:13.025563+0000","last_became_active":"2023-12-04T05:40:11.409814+0000","last_became_peered":"2023-12-04T05:40:11.409814+0000","last_unstale":"2023-12-04T05:40:13.025563+0000","last_undegraded":"2023-12-04T05:40:13.025563+0000","last_fullsized":"2023-12-04T05:40:13.025563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:58:01.481075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326574+0000","last_change":"2023-12-04T05:40:11.417351+0000","last_active":"2023-12-04T05:40:13.326574+0000","last_peered":"2023-12-04T05:40:13.326574+0000","last_clean":"2023-12-04T05:40:13.326574+0000","last_became_active":"2023-12-04T05:40:11.417095+0000","last_became_peered":"2023-12-04T05:40:11.417095+0000","last_unstale":"2023-12-04T05:40:13.326574+0000","last_undegraded":"2023-12-04T05:40:13.326574+0000","last_fullsized":"2023-12-04T05:40:13.326574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:10:38.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.399806+0000","last_change":"2023-12-04T05:40:11.405935+0000","last_active":"2023-12-04T05:40:12.399806+0000","last_peered":"2023-12-04T05:40:12.399806+0000","last_clean":"2023-12-04T05:40:12.399806+0000","last_became_active":"2023-12-04T05:40:11.402354+0000","last_became_peered":"2023-12-04T05:40:11.402354+0000","last_unstale":"2023-12-04T05:40:12.399806+0000","last_undegraded":"2023-12-04T05:40:12.399806+0000","last_fullsized":"2023-12-04T05:40:12.399806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:43:28.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.327088+0000","last_change":"2023-12-04T05:40:11.404034+0000","last_active":"2023-12-04T05:40:13.327088+0000","last_peered":"2023-12-04T05:40:13.327088+0000","last_clean":"2023-12-04T05:40:13.327088+0000","last_became_active":"2023-12-04T05:40:11.403714+0000","last_became_peered":"2023-12-04T05:40:11.403714+0000","last_unstale":"2023-12-04T05:40:13.327088+0000","last_undegraded":"2023-12-04T05:40:13.327088+0000","last_fullsized":"2023-12-04T05:40:13.327088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:03.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326564+0000","last_change":"2023-12-04T05:40:11.404640+0000","last_active":"2023-12-04T05:40:13.326564+0000","last_peered":"2023-12-04T05:40:13.326564+0000","last_clean":"2023-12-04T05:40:13.326564+0000","last_became_active":"2023-12-04T05:40:11.404392+0000","last_became_peered":"2023-12-04T05:40:11.404392+0000","last_unstale":"2023-12-04T05:40:13.326564+0000","last_undegraded":"2023-12-04T05:40:13.326564+0000","last_fullsized":"2023-12-04T05:40:13.326564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:44:28.365470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400644+0000","last_change":"2023-12-04T05:40:11.410366+0000","last_active":"2023-12-04T05:40:12.400644+0000","last_peered":"2023-12-04T05:40:12.400644+0000","last_clean":"2023-12-04T05:40:12.400644+0000","last_became_active":"2023-12-04T05:40:11.410095+0000","last_became_peered":"2023-12-04T05:40:11.410095+0000","last_unstale":"2023-12-04T05:40:12.400644+0000","last_undegraded":"2023-12-04T05:40:12.400644+0000","last_fullsized":"2023-12-04T05:40:12.400644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:42.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326659+0000","last_change":"2023-12-04T05:40:11.418191+0000","last_active":"2023-12-04T05:40:13.326659+0000","last_peered":"2023-12-04T05:40:13.326659+0000","last_clean":"2023-12-04T05:40:13.326659+0000","last_became_active":"2023-12-04T05:40:11.417874+0000","last_became_peered":"2023-12-04T05:40:11.417874+0000","last_unstale":"2023-12-04T05:40:13.326659+0000","last_undegraded":"2023-12-04T05:40:13.326659+0000","last_fullsized":"2023-12-04T05:40:13.326659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:44:28.365470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326748+0000","last_change":"2023-12-04T05:40:11.414358+0000","last_active":"2023-12-04T05:40:13.326748+0000","last_peered":"2023-12-04T05:40:13.326748+0000","last_clean":"2023-12-04T05:40:13.326748+0000","last_became_active":"2023-12-04T05:40:11.414063+0000","last_became_peered":"2023-12-04T05:40:11.414063+0000","last_unstale":"2023-12-04T05:40:13.326748+0000","last_undegraded":"2023-12-04T05:40:13.326748+0000","last_fullsized":"2023-12-04T05:40:13.326748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:42.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326861+0000","last_change":"2023-12-04T05:40:11.405285+0000","last_active":"2023-12-04T05:40:13.326861+0000","last_peered":"2023-12-04T05:40:13.326861+0000","last_clean":"2023-12-04T05:40:13.326861+0000","last_became_active":"2023-12-04T05:40:11.405016+0000","last_became_peered":"2023-12-04T05:40:11.405016+0000","last_unstale":"2023-12-04T05:40:13.326861+0000","last_undegraded":"2023-12-04T05:40:13.326861+0000","last_fullsized":"2023-12-04T05:40:13.326861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:12:51.236460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326338+0000","last_change":"2023-12-04T05:40:11.414341+0000","last_active":"2023-12-04T05:40:13.326338+0000","last_peered":"2023-12-04T05:40:13.326338+0000","last_clean":"2023-12-04T05:40:13.326338+0000","last_became_active":"2023-12-04T05:40:11.414054+0000","last_became_peered":"2023-12-04T05:40:11.414054+0000","last_unstale":"2023-12-04T05:40:13.326338+0000","last_undegraded":"2023-12-04T05:40:13.326338+0000","last_fullsized":"2023-12-04T05:40:13.326338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:10:38.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326387+0000","last_change":"2023-12-04T05:40:11.418259+0000","last_active":"2023-12-04T05:40:13.326387+0000","last_peered":"2023-12-04T05:40:13.326387+0000","last_clean":"2023-12-04T05:40:13.326387+0000","last_became_active":"2023-12-04T05:40:11.417969+0000","last_became_peered":"2023-12-04T05:40:11.417969+0000","last_unstale":"2023-12-04T05:40:13.326387+0000","last_undegraded":"2023-12-04T05:40:13.326387+0000","last_fullsized":"2023-12-04T05:40:13.326387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:21:54.092121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400834+0000","last_change":"2023-12-04T05:40:11.405956+0000","last_active":"2023-12-04T05:40:12.400834+0000","last_peered":"2023-12-04T05:40:12.400834+0000","last_clean":"2023-12-04T05:40:12.400834+0000","last_became_active":"2023-12-04T05:40:11.405701+0000","last_became_peered":"2023-12-04T05:40:11.405701+0000","last_unstale":"2023-12-04T05:40:12.400834+0000","last_undegraded":"2023-12-04T05:40:12.400834+0000","last_fullsized":"2023-12-04T05:40:12.400834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:21:16.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400918+0000","last_change":"2023-12-04T05:40:11.401467+0000","last_active":"2023-12-04T05:40:12.400918+0000","last_peered":"2023-12-04T05:40:12.400918+0000","last_clean":"2023-12-04T05:40:12.400918+0000","last_became_active":"2023-12-04T05:40:11.401176+0000","last_became_peered":"2023-12-04T05:40:11.401176+0000","last_unstale":"2023-12-04T05:40:12.400918+0000","last_undegraded":"2023-12-04T05:40:12.400918+0000","last_fullsized":"2023-12-04T05:40:12.400918+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:09:59.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398716+0000","last_change":"2023-12-04T05:40:11.413976+0000","last_active":"2023-12-04T05:40:12.398716+0000","last_peered":"2023-12-04T05:40:12.398716+0000","last_clean":"2023-12-04T05:40:12.398716+0000","last_became_active":"2023-12-04T05:40:11.413721+0000","last_became_peered":"2023-12-04T05:40:11.413721+0000","last_unstale":"2023-12-04T05:40:12.398716+0000","last_undegraded":"2023-12-04T05:40:12.398716+0000","last_fullsized":"2023-12-04T05:40:12.398716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:14.845449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400757+0000","last_change":"2023-12-04T05:40:11.405538+0000","last_active":"2023-12-04T05:40:12.400757+0000","last_peered":"2023-12-04T05:40:12.400757+0000","last_clean":"2023-12-04T05:40:12.400757+0000","last_became_active":"2023-12-04T05:40:11.405170+0000","last_became_peered":"2023-12-04T05:40:11.405170+0000","last_unstale":"2023-12-04T05:40:12.400757+0000","last_undegraded":"2023-12-04T05:40:12.400757+0000","last_fullsized":"2023-12-04T05:40:12.400757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:02.840080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025626+0000","last_change":"2023-12-04T05:40:11.409255+0000","last_active":"2023-12-04T05:40:13.025626+0000","last_peered":"2023-12-04T05:40:13.025626+0000","last_clean":"2023-12-04T05:40:13.025626+0000","last_became_active":"2023-12-04T05:40:11.408939+0000","last_became_peered":"2023-12-04T05:40:11.408939+0000","last_unstale":"2023-12-04T05:40:13.025626+0000","last_undegraded":"2023-12-04T05:40:13.025626+0000","last_fullsized":"2023-12-04T05:40:13.025626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:32:53.706261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326931+0000","last_change":"2023-12-04T05:40:11.407921+0000","last_active":"2023-12-04T05:40:13.326931+0000","last_peered":"2023-12-04T05:40:13.326931+0000","last_clean":"2023-12-04T05:40:13.326931+0000","last_became_active":"2023-12-04T05:40:11.407667+0000","last_became_peered":"2023-12-04T05:40:11.407667+0000","last_unstale":"2023-12-04T05:40:13.326931+0000","last_undegraded":"2023-12-04T05:40:13.326931+0000","last_fullsized":"2023-12-04T05:40:13.326931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:11:05.665982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025602+0000","last_change":"2023-12-04T05:40:11.415046+0000","last_active":"2023-12-04T05:40:13.025602+0000","last_peered":"2023-12-04T05:40:13.025602+0000","last_clean":"2023-12-04T05:40:13.025602+0000","last_became_active":"2023-12-04T05:40:11.414783+0000","last_became_peered":"2023-12-04T05:40:11.414783+0000","last_unstale":"2023-12-04T05:40:13.025602+0000","last_undegraded":"2023-12-04T05:40:13.025602+0000","last_fullsized":"2023-12-04T05:40:13.025602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:00:31.591764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.399967+0000","last_change":"2023-12-04T05:40:11.409823+0000","last_active":"2023-12-04T05:40:12.399967+0000","last_peered":"2023-12-04T05:40:12.399967+0000","last_clean":"2023-12-04T05:40:12.399967+0000","last_became_active":"2023-12-04T05:40:11.409567+0000","last_became_peered":"2023-12-04T05:40:11.409567+0000","last_unstale":"2023-12-04T05:40:12.399967+0000","last_undegraded":"2023-12-04T05:40:12.399967+0000","last_fullsized":"2023-12-04T05:40:12.399967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:58:01.481075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400784+0000","last_change":"2023-12-04T05:40:11.404475+0000","last_active":"2023-12-04T05:40:12.400784+0000","last_peered":"2023-12-04T05:40:12.400784+0000","last_clean":"2023-12-04T05:40:12.400784+0000","last_became_active":"2023-12-04T05:40:11.404163+0000","last_became_peered":"2023-12-04T05:40:11.404163+0000","last_unstale":"2023-12-04T05:40:12.400784+0000","last_undegraded":"2023-12-04T05:40:12.400784+0000","last_fullsized":"2023-12-04T05:40:12.400784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:29:23.930068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326388+0000","last_change":"2023-12-04T05:40:11.413141+0000","last_active":"2023-12-04T05:40:13.326388+0000","last_peered":"2023-12-04T05:40:13.326388+0000","last_clean":"2023-12-04T05:40:13.326388+0000","last_became_active":"2023-12-04T05:40:11.412847+0000","last_became_peered":"2023-12-04T05:40:11.412847+0000","last_unstale":"2023-12-04T05:40:13.326388+0000","last_undegraded":"2023-12-04T05:40:13.326388+0000","last_fullsized":"2023-12-04T05:40:13.326388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:43:28.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225085+0000","last_change":"2023-12-04T05:40:11.411190+0000","last_active":"2023-12-04T05:40:13.225085+0000","last_peered":"2023-12-04T05:40:13.225085+0000","last_clean":"2023-12-04T05:40:13.225085+0000","last_became_active":"2023-12-04T05:40:11.410785+0000","last_became_peered":"2023-12-04T05:40:11.410785+0000","last_unstale":"2023-12-04T05:40:13.225085+0000","last_undegraded":"2023-12-04T05:40:13.225085+0000","last_fullsized":"2023-12-04T05:40:13.225085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:43.525925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326221+0000","last_change":"2023-12-04T05:40:11.417572+0000","last_active":"2023-12-04T05:40:13.326221+0000","last_peered":"2023-12-04T05:40:13.326221+0000","last_clean":"2023-12-04T05:40:13.326221+0000","last_became_active":"2023-12-04T05:40:11.417232+0000","last_became_peered":"2023-12-04T05:40:11.417232+0000","last_unstale":"2023-12-04T05:40:13.326221+0000","last_undegraded":"2023-12-04T05:40:13.326221+0000","last_fullsized":"2023-12-04T05:40:13.326221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:09:59.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398870+0000","last_change":"2023-12-04T05:40:11.408047+0000","last_active":"2023-12-04T05:40:12.398870+0000","last_peered":"2023-12-04T05:40:12.398870+0000","last_clean":"2023-12-04T05:40:12.398870+0000","last_became_active":"2023-12-04T05:40:11.407789+0000","last_became_peered":"2023-12-04T05:40:11.407789+0000","last_unstale":"2023-12-04T05:40:12.398870+0000","last_undegraded":"2023-12-04T05:40:12.398870+0000","last_fullsized":"2023-12-04T05:40:12.398870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:11:51.087711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422526+0000","last_change":"2023-12-04T05:40:10.396846+0000","last_active":"2023-12-04T05:40:11.422526+0000","last_peered":"2023-12-04T05:40:11.422526+0000","last_clean":"2023-12-04T05:40:11.422526+0000","last_became_active":"2023-12-04T05:40:10.396580+0000","last_became_peered":"2023-12-04T05:40:10.396580+0000","last_unstale":"2023-12-04T05:40:11.422526+0000","last_undegraded":"2023-12-04T05:40:11.422526+0000","last_fullsized":"2023-12-04T05:40:11.422526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:16:48.177146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225370+0000","last_change":"2023-12-04T05:40:11.417343+0000","last_active":"2023-12-04T05:40:13.225370+0000","last_peered":"2023-12-04T05:40:13.225370+0000","last_clean":"2023-12-04T05:40:13.225370+0000","last_became_active":"2023-12-04T05:40:11.417036+0000","last_became_peered":"2023-12-04T05:40:11.417036+0000","last_unstale":"2023-12-04T05:40:13.225370+0000","last_undegraded":"2023-12-04T05:40:13.225370+0000","last_fullsized":"2023-12-04T05:40:13.225370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:38.464952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398834+0000","last_change":"2023-12-04T05:40:10.400176+0000","last_active":"2023-12-04T05:40:12.398834+0000","last_peered":"2023-12-04T05:40:12.398834+0000","last_clean":"2023-12-04T05:40:12.398834+0000","last_became_active":"2023-12-04T05:40:10.399818+0000","last_became_peered":"2023-12-04T05:40:10.399818+0000","last_unstale":"2023-12-04T05:40:12.398834+0000","last_undegraded":"2023-12-04T05:40:12.398834+0000","last_fullsized":"2023-12-04T05:40:12.398834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:42.521810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400971+0000","last_change":"2023-12-04T05:40:11.404798+0000","last_active":"2023-12-04T05:40:12.400971+0000","last_peered":"2023-12-04T05:40:12.400971+0000","last_clean":"2023-12-04T05:40:12.400971+0000","last_became_active":"2023-12-04T05:40:11.404294+0000","last_became_peered":"2023-12-04T05:40:11.404294+0000","last_unstale":"2023-12-04T05:40:12.400971+0000","last_undegraded":"2023-12-04T05:40:12.400971+0000","last_fullsized":"2023-12-04T05:40:12.400971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421369+0000","last_change":"2023-12-04T05:40:10.401657+0000","last_active":"2023-12-04T05:40:11.421369+0000","last_peered":"2023-12-04T05:40:11.421369+0000","last_clean":"2023-12-04T05:40:11.421369+0000","last_became_active":"2023-12-04T05:40:10.401385+0000","last_became_peered":"2023-12-04T05:40:10.401385+0000","last_unstale":"2023-12-04T05:40:11.421369+0000","last_undegraded":"2023-12-04T05:40:11.421369+0000","last_fullsized":"2023-12-04T05:40:11.421369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:20:48.384614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025630+0000","last_change":"2023-12-04T05:40:11.413857+0000","last_active":"2023-12-04T05:40:13.025630+0000","last_peered":"2023-12-04T05:40:13.025630+0000","last_clean":"2023-12-04T05:40:13.025630+0000","last_became_active":"2023-12-04T05:40:11.411905+0000","last_became_peered":"2023-12-04T05:40:11.411905+0000","last_unstale":"2023-12-04T05:40:13.025630+0000","last_undegraded":"2023-12-04T05:40:13.025630+0000","last_fullsized":"2023-12-04T05:40:13.025630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421180+0000","last_change":"2023-12-04T05:40:10.393071+0000","last_active":"2023-12-04T05:40:11.421180+0000","last_peered":"2023-12-04T05:40:11.421180+0000","last_clean":"2023-12-04T05:40:11.421180+0000","last_became_active":"2023-12-04T05:40:10.391756+0000","last_became_peered":"2023-12-04T05:40:10.391756+0000","last_unstale":"2023-12-04T05:40:11.421180+0000","last_undegraded":"2023-12-04T05:40:11.421180+0000","last_fullsized":"2023-12-04T05:40:11.421180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:22:57.770009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326395+0000","last_change":"2023-12-04T05:40:11.417955+0000","last_active":"2023-12-04T05:40:13.326395+0000","last_peered":"2023-12-04T05:40:13.326395+0000","last_clean":"2023-12-04T05:40:13.326395+0000","last_became_active":"2023-12-04T05:40:11.417629+0000","last_became_peered":"2023-12-04T05:40:11.417629+0000","last_unstale":"2023-12-04T05:40:13.326395+0000","last_undegraded":"2023-12-04T05:40:13.326395+0000","last_fullsized":"2023-12-04T05:40:13.326395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:58:01.481075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422068+0000","last_change":"2023-12-04T05:40:10.395884+0000","last_active":"2023-12-04T05:40:11.422068+0000","last_peered":"2023-12-04T05:40:11.422068+0000","last_clean":"2023-12-04T05:40:11.422068+0000","last_became_active":"2023-12-04T05:40:10.394514+0000","last_became_peered":"2023-12-04T05:40:10.394514+0000","last_unstale":"2023-12-04T05:40:11.422068+0000","last_undegraded":"2023-12-04T05:40:11.422068+0000","last_fullsized":"2023-12-04T05:40:11.422068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:21:53.088005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.399696+0000","last_change":"2023-12-04T05:40:11.409223+0000","last_active":"2023-12-04T05:40:12.399696+0000","last_peered":"2023-12-04T05:40:12.399696+0000","last_clean":"2023-12-04T05:40:12.399696+0000","last_became_active":"2023-12-04T05:40:11.408950+0000","last_became_peered":"2023-12-04T05:40:11.408950+0000","last_unstale":"2023-12-04T05:40:12.399696+0000","last_undegraded":"2023-12-04T05:40:12.399696+0000","last_fullsized":"2023-12-04T05:40:12.399696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421752+0000","last_change":"2023-12-04T05:40:10.395353+0000","last_active":"2023-12-04T05:40:11.421752+0000","last_peered":"2023-12-04T05:40:11.421752+0000","last_clean":"2023-12-04T05:40:11.421752+0000","last_became_active":"2023-12-04T05:40:10.395074+0000","last_became_peered":"2023-12-04T05:40:10.395074+0000","last_unstale":"2023-12-04T05:40:11.421752+0000","last_undegraded":"2023-12-04T05:40:11.421752+0000","last_fullsized":"2023-12-04T05:40:11.421752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:36:11.911650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400595+0000","last_change":"2023-12-04T05:40:11.409403+0000","last_active":"2023-12-04T05:40:12.400595+0000","last_peered":"2023-12-04T05:40:12.400595+0000","last_clean":"2023-12-04T05:40:12.400595+0000","last_became_active":"2023-12-04T05:40:11.409152+0000","last_became_peered":"2023-12-04T05:40:11.409152+0000","last_unstale":"2023-12-04T05:40:12.400595+0000","last_undegraded":"2023-12-04T05:40:12.400595+0000","last_fullsized":"2023-12-04T05:40:12.400595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:58:58.831382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.391773+0000","last_change":"2023-12-04T05:40:10.398424+0000","last_active":"2023-12-04T05:40:11.391773+0000","last_peered":"2023-12-04T05:40:11.391773+0000","last_clean":"2023-12-04T05:40:11.391773+0000","last_became_active":"2023-12-04T05:40:10.398186+0000","last_became_peered":"2023-12-04T05:40:10.398186+0000","last_unstale":"2023-12-04T05:40:11.391773+0000","last_undegraded":"2023-12-04T05:40:11.391773+0000","last_fullsized":"2023-12-04T05:40:11.391773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025189+0000","last_change":"2023-12-04T05:40:11.419281+0000","last_active":"2023-12-04T05:40:13.025189+0000","last_peered":"2023-12-04T05:40:13.025189+0000","last_clean":"2023-12-04T05:40:13.025189+0000","last_became_active":"2023-12-04T05:40:11.419019+0000","last_became_peered":"2023-12-04T05:40:11.419019+0000","last_unstale":"2023-12-04T05:40:13.025189+0000","last_undegraded":"2023-12-04T05:40:13.025189+0000","last_fullsized":"2023-12-04T05:40:13.025189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:21:16.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.394131+0000","last_change":"2023-12-04T05:40:10.393371+0000","last_active":"2023-12-04T05:40:11.394131+0000","last_peered":"2023-12-04T05:40:11.394131+0000","last_clean":"2023-12-04T05:40:11.394131+0000","last_became_active":"2023-12-04T05:40:10.393122+0000","last_became_peered":"2023-12-04T05:40:10.393122+0000","last_unstale":"2023-12-04T05:40:11.394131+0000","last_undegraded":"2023-12-04T05:40:11.394131+0000","last_fullsized":"2023-12-04T05:40:11.394131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:10:58.497714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400004+0000","last_change":"2023-12-04T05:40:11.409463+0000","last_active":"2023-12-04T05:40:12.400004+0000","last_peered":"2023-12-04T05:40:12.400004+0000","last_clean":"2023-12-04T05:40:12.400004+0000","last_became_active":"2023-12-04T05:40:11.409215+0000","last_became_peered":"2023-12-04T05:40:11.409215+0000","last_unstale":"2023-12-04T05:40:12.400004+0000","last_undegraded":"2023-12-04T05:40:12.400004+0000","last_fullsized":"2023-12-04T05:40:12.400004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:00:31.591764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.392193+0000","last_change":"2023-12-04T05:40:10.400963+0000","last_active":"2023-12-04T05:40:11.392193+0000","last_peered":"2023-12-04T05:40:11.392193+0000","last_clean":"2023-12-04T05:40:11.392193+0000","last_became_active":"2023-12-04T05:40:10.400686+0000","last_became_peered":"2023-12-04T05:40:10.400686+0000","last_unstale":"2023-12-04T05:40:11.392193+0000","last_undegraded":"2023-12-04T05:40:11.392193+0000","last_fullsized":"2023-12-04T05:40:11.392193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:58:57.827267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326870+0000","last_change":"2023-12-04T05:40:11.419905+0000","last_active":"2023-12-04T05:40:13.326870+0000","last_peered":"2023-12-04T05:40:13.326870+0000","last_clean":"2023-12-04T05:40:13.326870+0000","last_became_active":"2023-12-04T05:40:11.419620+0000","last_became_peered":"2023-12-04T05:40:11.419620+0000","last_unstale":"2023-12-04T05:40:13.326870+0000","last_undegraded":"2023-12-04T05:40:13.326870+0000","last_fullsized":"2023-12-04T05:40:13.326870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:19:19.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398915+0000","last_change":"2023-12-04T05:40:10.395994+0000","last_active":"2023-12-04T05:40:12.398915+0000","last_peered":"2023-12-04T05:40:12.398915+0000","last_clean":"2023-12-04T05:40:12.398915+0000","last_became_active":"2023-12-04T05:40:10.395726+0000","last_became_peered":"2023-12-04T05:40:10.395726+0000","last_unstale":"2023-12-04T05:40:12.398915+0000","last_undegraded":"2023-12-04T05:40:12.398915+0000","last_fullsized":"2023-12-04T05:40:12.398915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:39.600311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326610+0000","last_change":"2023-12-04T05:40:11.406971+0000","last_active":"2023-12-04T05:40:13.326610+0000","last_peered":"2023-12-04T05:40:13.326610+0000","last_clean":"2023-12-04T05:40:13.326610+0000","last_became_active":"2023-12-04T05:40:11.406713+0000","last_became_peered":"2023-12-04T05:40:11.406713+0000","last_unstale":"2023-12-04T05:40:13.326610+0000","last_undegraded":"2023-12-04T05:40:13.326610+0000","last_fullsized":"2023-12-04T05:40:13.326610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:42.028005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422652+0000","last_change":"2023-12-04T05:40:10.401157+0000","last_active":"2023-12-04T05:40:11.422652+0000","last_peered":"2023-12-04T05:40:11.422652+0000","last_clean":"2023-12-04T05:40:11.422652+0000","last_became_active":"2023-12-04T05:40:10.400828+0000","last_became_peered":"2023-12-04T05:40:10.400828+0000","last_unstale":"2023-12-04T05:40:11.422652+0000","last_undegraded":"2023-12-04T05:40:11.422652+0000","last_fullsized":"2023-12-04T05:40:11.422652+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:29:22.925953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400164+0000","last_change":"2023-12-04T05:40:11.409680+0000","last_active":"2023-12-04T05:40:12.400164+0000","last_peered":"2023-12-04T05:40:12.400164+0000","last_clean":"2023-12-04T05:40:12.400164+0000","last_became_active":"2023-12-04T05:40:11.409432+0000","last_became_peered":"2023-12-04T05:40:11.409432+0000","last_unstale":"2023-12-04T05:40:12.400164+0000","last_undegraded":"2023-12-04T05:40:12.400164+0000","last_fullsized":"2023-12-04T05:40:12.400164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.423675+0000","last_change":"2023-12-04T05:40:10.401551+0000","last_active":"2023-12-04T05:40:11.423675+0000","last_peered":"2023-12-04T05:40:11.423675+0000","last_clean":"2023-12-04T05:40:11.423675+0000","last_became_active":"2023-12-04T05:40:10.401316+0000","last_became_peered":"2023-12-04T05:40:10.401316+0000","last_unstale":"2023-12-04T05:40:11.423675+0000","last_undegraded":"2023-12-04T05:40:11.423675+0000","last_fullsized":"2023-12-04T05:40:11.423675+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:37.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025188+0000","last_change":"2023-12-04T05:40:11.419368+0000","last_active":"2023-12-04T05:40:13.025188+0000","last_peered":"2023-12-04T05:40:13.025188+0000","last_clean":"2023-12-04T05:40:13.025188+0000","last_became_active":"2023-12-04T05:40:11.419094+0000","last_became_peered":"2023-12-04T05:40:11.419094+0000","last_unstale":"2023-12-04T05:40:13.025188+0000","last_undegraded":"2023-12-04T05:40:13.025188+0000","last_fullsized":"2023-12-04T05:40:13.025188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:09:59.074502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422014+0000","last_change":"2023-12-04T05:40:10.390589+0000","last_active":"2023-12-04T05:40:11.422014+0000","last_peered":"2023-12-04T05:40:11.422014+0000","last_clean":"2023-12-04T05:40:11.422014+0000","last_became_active":"2023-12-04T05:40:10.390289+0000","last_became_peered":"2023-12-04T05:40:10.390289+0000","last_unstale":"2023-12-04T05:40:11.422014+0000","last_undegraded":"2023-12-04T05:40:11.422014+0000","last_fullsized":"2023-12-04T05:40:11.422014+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326635+0000","last_change":"2023-12-04T05:40:11.403109+0000","last_active":"2023-12-04T05:40:13.326635+0000","last_peered":"2023-12-04T05:40:13.326635+0000","last_clean":"2023-12-04T05:40:13.326635+0000","last_became_active":"2023-12-04T05:40:11.402842+0000","last_became_peered":"2023-12-04T05:40:11.402842+0000","last_unstale":"2023-12-04T05:40:13.326635+0000","last_undegraded":"2023-12-04T05:40:13.326635+0000","last_fullsized":"2023-12-04T05:40:13.326635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:02.840080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.402390+0000","last_change":"2023-12-04T05:40:10.397172+0000","last_active":"2023-12-04T05:40:11.402390+0000","last_peered":"2023-12-04T05:40:11.402390+0000","last_clean":"2023-12-04T05:40:11.402390+0000","last_became_active":"2023-12-04T05:40:10.396752+0000","last_became_peered":"2023-12-04T05:40:10.396752+0000","last_unstale":"2023-12-04T05:40:11.402390+0000","last_undegraded":"2023-12-04T05:40:11.402390+0000","last_fullsized":"2023-12-04T05:40:11.402390+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:39.600311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326477+0000","last_change":"2023-12-04T05:40:11.417904+0000","last_active":"2023-12-04T05:40:13.326477+0000","last_peered":"2023-12-04T05:40:13.326477+0000","last_clean":"2023-12-04T05:40:13.326477+0000","last_became_active":"2023-12-04T05:40:11.417618+0000","last_became_peered":"2023-12-04T05:40:11.417618+0000","last_unstale":"2023-12-04T05:40:13.326477+0000","last_undegraded":"2023-12-04T05:40:13.326477+0000","last_fullsized":"2023-12-04T05:40:13.326477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:00:31.591764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.391852+0000","last_change":"2023-12-04T05:40:10.399192+0000","last_active":"2023-12-04T05:40:11.391852+0000","last_peered":"2023-12-04T05:40:11.391852+0000","last_clean":"2023-12-04T05:40:11.391852+0000","last_became_active":"2023-12-04T05:40:10.398915+0000","last_became_peered":"2023-12-04T05:40:10.398915+0000","last_unstale":"2023-12-04T05:40:11.391852+0000","last_undegraded":"2023-12-04T05:40:11.391852+0000","last_fullsized":"2023-12-04T05:40:11.391852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:42.521810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225205+0000","last_change":"2023-12-04T05:40:11.417361+0000","last_active":"2023-12-04T05:40:13.225205+0000","last_peered":"2023-12-04T05:40:13.225205+0000","last_clean":"2023-12-04T05:40:13.225205+0000","last_became_active":"2023-12-04T05:40:11.417067+0000","last_became_peered":"2023-12-04T05:40:11.417067+0000","last_unstale":"2023-12-04T05:40:13.225205+0000","last_undegraded":"2023-12-04T05:40:13.225205+0000","last_fullsized":"2023-12-04T05:40:13.225205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:39:57.822988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421998+0000","last_change":"2023-12-04T05:40:10.401519+0000","last_active":"2023-12-04T05:40:11.421998+0000","last_peered":"2023-12-04T05:40:11.421998+0000","last_clean":"2023-12-04T05:40:11.421998+0000","last_became_active":"2023-12-04T05:40:10.401274+0000","last_became_peered":"2023-12-04T05:40:10.401274+0000","last_unstale":"2023-12-04T05:40:11.421998+0000","last_undegraded":"2023-12-04T05:40:11.421998+0000","last_fullsized":"2023-12-04T05:40:11.421998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400161+0000","last_change":"2023-12-04T05:40:11.410967+0000","last_active":"2023-12-04T05:40:12.400161+0000","last_peered":"2023-12-04T05:40:12.400161+0000","last_clean":"2023-12-04T05:40:12.400161+0000","last_became_active":"2023-12-04T05:40:11.410547+0000","last_became_peered":"2023-12-04T05:40:11.410547+0000","last_unstale":"2023-12-04T05:40:12.400161+0000","last_undegraded":"2023-12-04T05:40:12.400161+0000","last_fullsized":"2023-12-04T05:40:12.400161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:38.464952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421450+0000","last_change":"2023-12-04T05:40:10.397087+0000","last_active":"2023-12-04T05:40:11.421450+0000","last_peered":"2023-12-04T05:40:11.421450+0000","last_clean":"2023-12-04T05:40:11.421450+0000","last_became_active":"2023-12-04T05:40:10.396853+0000","last_became_peered":"2023-12-04T05:40:10.396853+0000","last_unstale":"2023-12-04T05:40:11.421450+0000","last_undegraded":"2023-12-04T05:40:11.421450+0000","last_fullsized":"2023-12-04T05:40:11.421450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:37.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400051+0000","last_change":"2023-12-04T05:40:11.403287+0000","last_active":"2023-12-04T05:40:12.400051+0000","last_peered":"2023-12-04T05:40:12.400051+0000","last_clean":"2023-12-04T05:40:12.400051+0000","last_became_active":"2023-12-04T05:40:11.403045+0000","last_became_peered":"2023-12-04T05:40:11.403045+0000","last_unstale":"2023-12-04T05:40:12.400051+0000","last_undegraded":"2023-12-04T05:40:12.400051+0000","last_fullsized":"2023-12-04T05:40:12.400051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:32:53.706261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421653+0000","last_change":"2023-12-04T05:40:10.391094+0000","last_active":"2023-12-04T05:40:11.421653+0000","last_peered":"2023-12-04T05:40:11.421653+0000","last_clean":"2023-12-04T05:40:11.421653+0000","last_became_active":"2023-12-04T05:40:10.390763+0000","last_became_peered":"2023-12-04T05:40:10.390763+0000","last_unstale":"2023-12-04T05:40:11.421653+0000","last_undegraded":"2023-12-04T05:40:11.421653+0000","last_fullsized":"2023-12-04T05:40:11.421653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:13.841334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.025362+0000","last_change":"2023-12-04T05:40:11.419232+0000","last_active":"2023-12-04T05:40:13.025362+0000","last_peered":"2023-12-04T05:40:13.025362+0000","last_clean":"2023-12-04T05:40:13.025362+0000","last_became_active":"2023-12-04T05:40:11.418953+0000","last_became_peered":"2023-12-04T05:40:11.418953+0000","last_unstale":"2023-12-04T05:40:13.025362+0000","last_undegraded":"2023-12-04T05:40:13.025362+0000","last_fullsized":"2023-12-04T05:40:13.025362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:43:28.108780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.421528+0000","last_change":"2023-12-04T05:40:10.395628+0000","last_active":"2023-12-04T05:40:11.421528+0000","last_peered":"2023-12-04T05:40:11.421528+0000","last_clean":"2023-12-04T05:40:11.421528+0000","last_became_active":"2023-12-04T05:40:10.395276+0000","last_became_peered":"2023-12-04T05:40:10.395276+0000","last_unstale":"2023-12-04T05:40:11.421528+0000","last_undegraded":"2023-12-04T05:40:11.421528+0000","last_fullsized":"2023-12-04T05:40:11.421528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326291+0000","last_change":"2023-12-04T05:40:11.408455+0000","last_active":"2023-12-04T05:40:13.326291+0000","last_peered":"2023-12-04T05:40:13.326291+0000","last_clean":"2023-12-04T05:40:13.326291+0000","last_became_active":"2023-12-04T05:40:11.408109+0000","last_became_peered":"2023-12-04T05:40:11.408109+0000","last_unstale":"2023-12-04T05:40:13.326291+0000","last_undegraded":"2023-12-04T05:40:13.326291+0000","last_fullsized":"2023-12-04T05:40:13.326291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:36:12.915765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.395872+0000","last_change":"2023-12-04T05:40:10.393607+0000","last_active":"2023-12-04T05:40:11.395872+0000","last_peered":"2023-12-04T05:40:11.395872+0000","last_clean":"2023-12-04T05:40:11.395872+0000","last_became_active":"2023-12-04T05:40:10.393352+0000","last_became_peered":"2023-12-04T05:40:10.393352+0000","last_unstale":"2023-12-04T05:40:11.395872+0000","last_undegraded":"2023-12-04T05:40:11.395872+0000","last_fullsized":"2023-12-04T05:40:11.395872+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:39.600311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326197+0000","last_change":"2023-12-04T05:40:11.413442+0000","last_active":"2023-12-04T05:40:13.326197+0000","last_peered":"2023-12-04T05:40:13.326197+0000","last_clean":"2023-12-04T05:40:13.326197+0000","last_became_active":"2023-12-04T05:40:11.413182+0000","last_became_peered":"2023-12-04T05:40:11.413182+0000","last_unstale":"2023-12-04T05:40:13.326197+0000","last_undegraded":"2023-12-04T05:40:13.326197+0000","last_fullsized":"2023-12-04T05:40:13.326197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:43:10.132992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398884+0000","last_change":"2023-12-04T05:40:10.396044+0000","last_active":"2023-12-04T05:40:12.398884+0000","last_peered":"2023-12-04T05:40:12.398884+0000","last_clean":"2023-12-04T05:40:12.398884+0000","last_became_active":"2023-12-04T05:40:10.395788+0000","last_became_peered":"2023-12-04T05:40:10.395788+0000","last_unstale":"2023-12-04T05:40:12.398884+0000","last_undegraded":"2023-12-04T05:40:12.398884+0000","last_fullsized":"2023-12-04T05:40:12.398884+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.327111+0000","last_change":"2023-12-04T05:40:11.412904+0000","last_active":"2023-12-04T05:40:13.327111+0000","last_peered":"2023-12-04T05:40:13.327111+0000","last_clean":"2023-12-04T05:40:13.327111+0000","last_became_active":"2023-12-04T05:40:11.412643+0000","last_became_peered":"2023-12-04T05:40:11.412643+0000","last_unstale":"2023-12-04T05:40:13.327111+0000","last_undegraded":"2023-12-04T05:40:13.327111+0000","last_fullsized":"2023-12-04T05:40:13.327111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:05:43.525925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.403723+0000","last_change":"2023-12-04T05:40:10.397551+0000","last_active":"2023-12-04T05:40:11.403723+0000","last_peered":"2023-12-04T05:40:11.403723+0000","last_clean":"2023-12-04T05:40:11.403723+0000","last_became_active":"2023-12-04T05:40:10.397198+0000","last_became_peered":"2023-12-04T05:40:10.397198+0000","last_unstale":"2023-12-04T05:40:11.403723+0000","last_undegraded":"2023-12-04T05:40:11.403723+0000","last_fullsized":"2023-12-04T05:40:11.403723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:22:51.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326262+0000","last_change":"2023-12-04T05:40:11.410058+0000","last_active":"2023-12-04T05:40:13.326262+0000","last_peered":"2023-12-04T05:40:13.326262+0000","last_clean":"2023-12-04T05:40:13.326262+0000","last_became_active":"2023-12-04T05:40:11.409790+0000","last_became_peered":"2023-12-04T05:40:11.409790+0000","last_unstale":"2023-12-04T05:40:13.326262+0000","last_undegraded":"2023-12-04T05:40:13.326262+0000","last_fullsized":"2023-12-04T05:40:13.326262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:03:44.891656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.393250+0000","last_change":"2023-12-04T05:40:10.392389+0000","last_active":"2023-12-04T05:40:11.393250+0000","last_peered":"2023-12-04T05:40:11.393250+0000","last_clean":"2023-12-04T05:40:11.393250+0000","last_became_active":"2023-12-04T05:40:10.390893+0000","last_became_peered":"2023-12-04T05:40:10.390893+0000","last_unstale":"2023-12-04T05:40:11.393250+0000","last_undegraded":"2023-12-04T05:40:11.393250+0000","last_fullsized":"2023-12-04T05:40:11.393250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:12:01.216530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326307+0000","last_change":"2023-12-04T05:40:11.413170+0000","last_active":"2023-12-04T05:40:13.326307+0000","last_peered":"2023-12-04T05:40:13.326307+0000","last_clean":"2023-12-04T05:40:13.326307+0000","last_became_active":"2023-12-04T05:40:11.412875+0000","last_became_peered":"2023-12-04T05:40:11.412875+0000","last_unstale":"2023-12-04T05:40:13.326307+0000","last_undegraded":"2023-12-04T05:40:13.326307+0000","last_fullsized":"2023-12-04T05:40:13.326307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:11:58.597711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.393035+0000","last_change":"2023-12-04T05:40:10.400242+0000","last_active":"2023-12-04T05:40:11.393035+0000","last_peered":"2023-12-04T05:40:11.393035+0000","last_clean":"2023-12-04T05:40:11.393035+0000","last_became_active":"2023-12-04T05:40:10.399939+0000","last_became_peered":"2023-12-04T05:40:10.399939+0000","last_unstale":"2023-12-04T05:40:11.393035+0000","last_undegraded":"2023-12-04T05:40:11.393035+0000","last_fullsized":"2023-12-04T05:40:11.393035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326263+0000","last_change":"2023-12-04T05:40:11.417608+0000","last_active":"2023-12-04T05:40:13.326263+0000","last_peered":"2023-12-04T05:40:13.326263+0000","last_clean":"2023-12-04T05:40:13.326263+0000","last_became_active":"2023-12-04T05:40:11.417350+0000","last_became_peered":"2023-12-04T05:40:11.417350+0000","last_unstale":"2023-12-04T05:40:13.326263+0000","last_undegraded":"2023-12-04T05:40:13.326263+0000","last_fullsized":"2023-12-04T05:40:13.326263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:16:49.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.422193+0000","last_change":"2023-12-04T05:40:10.396052+0000","last_active":"2023-12-04T05:40:11.422193+0000","last_peered":"2023-12-04T05:40:11.422193+0000","last_clean":"2023-12-04T05:40:11.422193+0000","last_became_active":"2023-12-04T05:40:10.395808+0000","last_became_peered":"2023-12-04T05:40:10.395808+0000","last_unstale":"2023-12-04T05:40:11.422193+0000","last_undegraded":"2023-12-04T05:40:11.422193+0000","last_fullsized":"2023-12-04T05:40:11.422193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:11:50.083596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400223+0000","last_change":"2023-12-04T05:40:11.409215+0000","last_active":"2023-12-04T05:40:12.400223+0000","last_peered":"2023-12-04T05:40:12.400223+0000","last_clean":"2023-12-04T05:40:12.400223+0000","last_became_active":"2023-12-04T05:40:11.408937+0000","last_became_peered":"2023-12-04T05:40:11.408937+0000","last_unstale":"2023-12-04T05:40:12.400223+0000","last_undegraded":"2023-12-04T05:40:12.400223+0000","last_fullsized":"2023-12-04T05:40:12.400223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:10:38.785141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398949+0000","last_change":"2023-12-04T05:40:10.396822+0000","last_active":"2023-12-04T05:40:12.398949+0000","last_peered":"2023-12-04T05:40:12.398949+0000","last_clean":"2023-12-04T05:40:12.398949+0000","last_became_active":"2023-12-04T05:40:10.396542+0000","last_became_peered":"2023-12-04T05:40:10.396542+0000","last_unstale":"2023-12-04T05:40:12.398949+0000","last_undegraded":"2023-12-04T05:40:12.398949+0000","last_fullsized":"2023-12-04T05:40:12.398949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:39:56.818872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398792+0000","last_change":"2023-12-04T05:40:11.415224+0000","last_active":"2023-12-04T05:40:12.398792+0000","last_peered":"2023-12-04T05:40:12.398792+0000","last_clean":"2023-12-04T05:40:12.398792+0000","last_became_active":"2023-12-04T05:40:11.414961+0000","last_became_peered":"2023-12-04T05:40:11.414961+0000","last_unstale":"2023-12-04T05:40:12.398792+0000","last_undegraded":"2023-12-04T05:40:12.398792+0000","last_fullsized":"2023-12-04T05:40:12.398792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:24:38.464952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.408480+0000","last_change":"2023-12-04T05:40:10.394243+0000","last_active":"2023-12-04T05:40:11.408480+0000","last_peered":"2023-12-04T05:40:11.408480+0000","last_clean":"2023-12-04T05:40:11.408480+0000","last_became_active":"2023-12-04T05:40:10.393531+0000","last_became_peered":"2023-12-04T05:40:10.393531+0000","last_unstale":"2023-12-04T05:40:11.408480+0000","last_undegraded":"2023-12-04T05:40:11.408480+0000","last_fullsized":"2023-12-04T05:40:11.408480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:20:48.384614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.225227+0000","last_change":"2023-12-04T05:40:11.412378+0000","last_active":"2023-12-04T05:40:13.225227+0000","last_peered":"2023-12-04T05:40:13.225227+0000","last_clean":"2023-12-04T05:40:13.225227+0000","last_became_active":"2023-12-04T05:40:11.412086+0000","last_became_peered":"2023-12-04T05:40:11.412086+0000","last_unstale":"2023-12-04T05:40:13.225227+0000","last_undegraded":"2023-12-04T05:40:13.225227+0000","last_fullsized":"2023-12-04T05:40:13.225227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:40.604427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398963+0000","last_change":"2023-12-04T05:40:10.397176+0000","last_active":"2023-12-04T05:40:12.398963+0000","last_peered":"2023-12-04T05:40:12.398963+0000","last_clean":"2023-12-04T05:40:12.398963+0000","last_became_active":"2023-12-04T05:40:10.396863+0000","last_became_peered":"2023-12-04T05:40:10.396863+0000","last_unstale":"2023-12-04T05:40:12.398963+0000","last_undegraded":"2023-12-04T05:40:12.398963+0000","last_fullsized":"2023-12-04T05:40:12.398963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:14:02.116527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.398891+0000","last_change":"2023-12-04T05:40:11.411082+0000","last_active":"2023-12-04T05:40:12.398891+0000","last_peered":"2023-12-04T05:40:12.398891+0000","last_clean":"2023-12-04T05:40:12.398891+0000","last_became_active":"2023-12-04T05:40:11.410549+0000","last_became_peered":"2023-12-04T05:40:11.410549+0000","last_unstale":"2023-12-04T05:40:12.398891+0000","last_undegraded":"2023-12-04T05:40:12.398891+0000","last_fullsized":"2023-12-04T05:40:12.398891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:58:58.831382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.402499+0000","last_change":"2023-12-04T05:40:10.397000+0000","last_active":"2023-12-04T05:40:11.402499+0000","last_peered":"2023-12-04T05:40:11.402499+0000","last_clean":"2023-12-04T05:40:11.402499+0000","last_became_active":"2023-12-04T05:40:10.396440+0000","last_became_peered":"2023-12-04T05:40:10.396440+0000","last_unstale":"2023-12-04T05:40:11.402499+0000","last_undegraded":"2023-12-04T05:40:11.402499+0000","last_fullsized":"2023-12-04T05:40:11.402499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:40:13.841334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326100+0000","last_change":"2023-12-04T05:40:11.405888+0000","last_active":"2023-12-04T05:40:13.326100+0000","last_peered":"2023-12-04T05:40:13.326100+0000","last_clean":"2023-12-04T05:40:13.326100+0000","last_became_active":"2023-12-04T05:40:11.405634+0000","last_became_peered":"2023-12-04T05:40:11.405634+0000","last_unstale":"2023-12-04T05:40:13.326100+0000","last_undegraded":"2023-12-04T05:40:13.326100+0000","last_fullsized":"2023-12-04T05:40:13.326100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.394201+0000","last_change":"2023-12-04T05:40:10.392063+0000","last_active":"2023-12-04T05:40:11.394201+0000","last_peered":"2023-12-04T05:40:11.394201+0000","last_clean":"2023-12-04T05:40:11.394201+0000","last_became_active":"2023-12-04T05:40:10.391681+0000","last_became_peered":"2023-12-04T05:40:10.391681+0000","last_unstale":"2023-12-04T05:40:11.394201+0000","last_undegraded":"2023-12-04T05:40:11.394201+0000","last_fullsized":"2023-12-04T05:40:11.394201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:22:57.770009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:12.400767+0000","last_change":"2023-12-04T05:40:11.403813+0000","last_active":"2023-12-04T05:40:12.400767+0000","last_peered":"2023-12-04T05:40:12.400767+0000","last_clean":"2023-12-04T05:40:12.400767+0000","last_became_active":"2023-12-04T05:40:11.402867+0000","last_became_peered":"2023-12-04T05:40:11.402867+0000","last_unstale":"2023-12-04T05:40:12.400767+0000","last_undegraded":"2023-12-04T05:40:12.400767+0000","last_fullsized":"2023-12-04T05:40:12.400767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:20:49.388730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.391893+0000","last_change":"2023-12-04T05:40:10.400004+0000","last_active":"2023-12-04T05:40:11.391893+0000","last_peered":"2023-12-04T05:40:11.391893+0000","last_clean":"2023-12-04T05:40:11.391893+0000","last_became_active":"2023-12-04T05:40:10.399698+0000","last_became_peered":"2023-12-04T05:40:10.399698+0000","last_unstale":"2023-12-04T05:40:11.391893+0000","last_undegraded":"2023-12-04T05:40:11.391893+0000","last_fullsized":"2023-12-04T05:40:11.391893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:39:56.818872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-04T05:40:13.326513+0000","last_change":"2023-12-04T05:40:11.412217+0000","last_active":"2023-12-04T05:40:13.326513+0000","last_peered":"2023-12-04T05:40:13.326513+0000","last_clean":"2023-12-04T05:40:13.326513+0000","last_became_active":"2023-12-04T05:40:11.411959+0000","last_became_peered":"2023-12-04T05:40:11.411959+0000","last_unstale":"2023-12-04T05:40:13.326513+0000","last_undegraded":"2023-12-04T05:40:13.326513+0000","last_fullsized":"2023-12-04T05:40:13.326513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:10.377744+0000","last_clean_scrub_stamp":"2023-12-04T05:40:10.377744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:45:37.291134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.403135+0000","last_change":"2023-12-04T05:40:10.398575+0000","last_active":"2023-12-04T05:40:11.403135+0000","last_peered":"2023-12-04T05:40:11.403135+0000","last_clean":"2023-12-04T05:40:11.403135+0000","last_became_active":"2023-12-04T05:40:10.398318+0000","last_became_peered":"2023-12-04T05:40:10.398318+0000","last_unstale":"2023-12-04T05:40:11.403135+0000","last_undegraded":"2023-12-04T05:40:11.403135+0000","last_fullsized":"2023-12-04T05:40:11.403135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:40:09.373629+0000","last_clean_scrub_stamp":"2023-12-04T05:40:09.373629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:36:11.911650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-04T05:40:11.405851+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:40:11.405851+0000","last_peered":"2023-12-04T05:40:11.405851+0000","last_clean":"2023-12-04T05:40:11.405851+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:40:11.405851+0000","last_undegraded":"2023-12-04T05:40:11.405851+0000","last_fullsized":"2023-12-04T05:40:11.405851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.435}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.5}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44500000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49199999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69699999999999995}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.748}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72799999999999998}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.627},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.51900000000000002}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.372},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.311}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48599999999999999}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.498}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51400000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.26700000000000002}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.314}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33000000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.498},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67400000000000004}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.35499999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":736952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.36899999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.311},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.40000000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.871}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:40:20.045 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-04T05:40:20.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-04T05:40:20.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.182+0000 7fc5659b0640 1 Processor -- start 2023-12-04T05:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.182+0000 7fc5659b0640 1 -- start start 2023-12-04T05:40:20.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc5659b0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc56015d330 0x7fc5600ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc5659b0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5600d1220 0x7fc5600b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc5659b0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5600b55b0 con 0x7fc5600b5170 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc5659b0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5600d2360 con 0x7fc5600d1220 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc5659b0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015d710 con 0x7fc56015d330 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55e7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5600d1220 0x7fc5600b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc5600b5170 0x7fc5600b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48830/0 (socket says 172.21.15.111:48830) 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55f7fe640 1 -- 172.21.15.111:0/3667429132 learned_addr learned my addr 172.21.15.111:0/3667429132 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55e7fc640 1 -- 172.21.15.111:0/3667429132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc56015d330 msgr2=0x7fc5600ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55e7fc640 1 --2- 172.21.15.111:0/3667429132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc56015d330 0x7fc5600ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55e7fc640 1 -- 172.21.15.111:0/3667429132 >> v1:172.21.15.111:6789/0 conn(0x7fc5600b5170 legacy=0x7fc5600b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55e7fc640 1 -- 172.21.15.111:0/3667429132 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5600d29b0 con 0x7fc5600d1220 2023-12-04T05:40:20.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.183+0000 7fc55dffb640 1 -- 172.21.15.111:0/3667429132 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995042246 0 0) 0x7fc5600b55b0 con 0x7fc5600b5170 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.184+0000 7fc55e7fc640 1 --2- 172.21.15.111:0/3667429132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600d1220 0x7fc5600b4c30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc5540097a0 tx=0x7fc55402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c6f4ee88db2814 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.184+0000 7fc55dffb640 1 -- 172.21.15.111:0/3667429132 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554031830 con 0x7fc5600d1220 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.184+0000 7fc55dffb640 1 -- 172.21.15.111:0/3667429132 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc554031990 con 0x7fc5600d1220 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.184+0000 7fc55dffb640 1 -- 172.21.15.111:0/3667429132 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554031d20 con 0x7fc5600d1220 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 -- 172.21.15.111:0/3667429132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600d1220 msgr2=0x7fc5600b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 --2- 172.21.15.111:0/3667429132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600d1220 0x7fc5600b4c30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc5540097a0 tx=0x7fc55402b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 -- 172.21.15.111:0/3667429132 shutdown_connections 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 --2- 172.21.15.111:0/3667429132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600d1220 0x7fc5600b4c30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 --2- 172.21.15.111:0/3667429132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc56015d330 0x7fc5600ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 -- 172.21.15.111:0/3667429132 >> 172.21.15.111:0/3667429132 conn(0x7fc560072220 msgr2=0x7fc5600cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 -- 172.21.15.111:0/3667429132 shutdown_connections 2023-12-04T05:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.185+0000 7fc5659b0640 1 -- 172.21.15.111:0/3667429132 wait complete. 2023-12-04T05:40:20.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.186+0000 7fc5659b0640 1 Processor -- start 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.186+0000 7fc5659b0640 1 -- start start 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 0x7fc5601599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600d1220 0x7fc5600b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc56015d330 0x7fc56015ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600a9ba0 con 0x7fc5600b5170 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc560159f10 con 0x7fc56015d330 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600cb1c0 con 0x7fc5600d1220 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 0x7fc5601599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 0x7fc5601599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42200/0 (socket says 172.21.15.111:42200) 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 -- 172.21.15.111:0/1911023698 learned_addr learned my addr 172.21.15.111:0/1911023698 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:20.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55f7fe640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc56015d330 0x7fc56015ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55e7fc640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600d1220 0x7fc5600b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 -- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600d1220 msgr2=0x7fc5600b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600d1220 0x7fc5600b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 -- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc56015d330 msgr2=0x7fc56015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc56015d330 0x7fc56015ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 -- 172.21.15.111:0/1911023698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc554009400 con 0x7fc5600b5170 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55f7fe640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc56015d330 0x7fc56015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55e7fc640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600d1220 0x7fc5600b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc55effd640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 0x7fc5601599d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc55000ba50 tx=0x7fc55000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:20.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000cbe0 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55000cd60 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5600ca5b0 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550014680 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.187+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5600cab70 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.188+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5600cb610 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.189+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc55001d3b0 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.189+0000 7fc547fff640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc54c06b2a0 0x7fc54c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.189+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc550089bf0 con 0x7fc5600b5170 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.189+0000 7fc55e7fc640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc54c06b2a0 0x7fc54c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.190+0000 7fc55e7fc640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc54c06b2a0 0x7fc54c06d760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc554036000 tx=0x7fc554008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:20.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.193+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5500549e0 con 0x7fc5600b5170 2023-12-04T05:40:20.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.424+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc5600693d0 con 0x7fc5600b5170 2023-12-04T05:40:20.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.428+0000 7fc547fff640 1 -- 172.21.15.111:0/1911023698 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc550058000 con 0x7fc5600b5170 2023-12-04T05:40:20.430 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2023-12-04T05:40:20.430 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDLZW1l+5T3ABAA7h+0Ksz9xnrfFZKgsBbhIA== 2023-12-04T05:40:20.430 INFO:teuthology.orchestra.run.smithi111.stdout: caps mds = "allow" 2023-12-04T05:40:20.431 INFO:teuthology.orchestra.run.smithi111.stdout: caps mon = "allow r" 2023-12-04T05:40:20.431 INFO:teuthology.orchestra.run.smithi111.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-04T05:40:20.431 INFO:teuthology.orchestra.run.smithi111.stderr:updated caps for client.0 2023-12-04T05:40:20.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc54c06b2a0 msgr2=0x7fc54c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc54c06b2a0 0x7fc54c06d760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc554036000 tx=0x7fc554008030 comp rx=0 tx=0).stop 2023-12-04T05:40:20.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 msgr2=0x7fc5601599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 0x7fc5601599d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc55000ba50 tx=0x7fc55000bf10 comp rx=0 tx=0).stop 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 shutdown_connections 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc54c06b2a0 0x7fc54c06d760 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc56015d330 0x7fc56015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600d1220 0x7fc5600b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 --2- 172.21.15.111:0/1911023698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5600b5170 0x7fc5601599d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 >> 172.21.15.111:0/1911023698 conn(0x7fc560072220 msgr2=0x7fc56015c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 shutdown_connections 2023-12-04T05:40:20.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.431+0000 7fc5659b0640 1 -- 172.21.15.111:0/1911023698 wait complete. 2023-12-04T05:40:20.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:20.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.541+0000 7f2888fec640 1 Processor -- start 2023-12-04T05:40:20.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.541+0000 7f2888fec640 1 -- start start 2023-12-04T05:40:20.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2888fec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f288415d510 0x7f28840d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2888fec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 0x7f28840d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2888fec640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840ce110 con 0x7f28840cfc00 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2888fec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28840ceab0 con 0x7f288415d510 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2888fec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ce3f0 con 0x7f28840ceed0 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2882d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f28840cfc00 0x7f28840d0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58198/0 (socket says 172.21.15.111:58198) 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.542+0000 7f2882d9d640 1 -- 172.21.15.111:0/218605230 learned_addr learned my addr 172.21.15.111:0/218605230 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2881d9b640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 0x7f28840d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288259c640 1 --2- 172.21.15.111:0/218605230 >> v2:172.21.15.192:3300/0 conn(0x7f288415d510 0x7f28840d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107073025 0 0) 0x7f28840ce110 con 0x7f28840cfc00 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f286c003520 con 0x7f28840cfc00 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3295562236 0 0) 0x7f286c003520 con 0x7f28840cfc00 2023-12-04T05:40:20.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 msgr2=0x7f28840d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 0x7f28840d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 >> v2:172.21.15.192:3300/0 conn(0x7f288415d510 msgr2=0x7f28840d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 --2- 172.21.15.111:0/218605230 >> v2:172.21.15.192:3300/0 conn(0x7f288415d510 0x7f28840d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840ce6d0 con 0x7f28840cfc00 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f287c001670 con 0x7f28840cfc00 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 >> v1:172.21.15.111:6789/0 conn(0x7f28840cfc00 legacy=0x7f28840d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 0x7f286c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f286c003ef0 0x7f286c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f286c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ceab0 con 0x7f286c004d00 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ce3f0 con 0x7f286c003ef0 2023-12-04T05:40:20.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286c00b980 con 0x7f28840ceed0 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2882d9d640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f286c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f288259c640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 0x7f286c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2881d9b640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f286c003ef0 0x7f286c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2882d9d640 1 -- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 msgr2=0x7f286c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2882d9d640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840ceed0 0x7f286c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2882d9d640 1 -- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f286c003ef0 msgr2=0x7f286c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2882d9d640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f286c003ef0 0x7f286c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.543+0000 7f2882d9d640 1 -- 172.21.15.111:0/218605230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c003520 con 0x7f286c004d00 2023-12-04T05:40:20.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.544+0000 7f2882d9d640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f286c005180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c240 tx=0x7f286c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1542ef7365e1a959 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:20.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.544+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c02cae0 con 0x7f286c004d00 2023-12-04T05:40:20.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.544+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c02cc40 con 0x7f286c004d00 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.544+0000 7f288159a640 1 -- 172.21.15.111:0/218605230 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c005af0 con 0x7f286c004d00 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2888fec640 1 -- 172.21.15.111:0/218605230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 msgr2=0x7f286c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2888fec640 1 --2- 172.21.15.111:0/218605230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f286c005180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c240 tx=0x7f286c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2882d9d640 1 -- 172.21.15.111:0/218605230 reap_dead start 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2888fec640 1 -- 172.21.15.111:0/218605230 shutdown_connections 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2888fec640 1 -- 172.21.15.111:0/218605230 >> 172.21.15.111:0/218605230 conn(0x7f288409c460 msgr2=0x7f28840d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2888fec640 1 -- 172.21.15.111:0/218605230 shutdown_connections 2023-12-04T05:40:20.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.545+0000 7f2888fec640 1 -- 172.21.15.111:0/218605230 wait complete. 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.546+0000 7f2888fec640 1 Processor -- start 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.546+0000 7f2888fec640 1 -- start start 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f2888fec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f28841596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f2888fec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28840b8360 0x7f28840b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f2888fec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840c7210 0x7f28840c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f2888fec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840b8d00 con 0x7f286c004d00 2023-12-04T05:40:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f2888fec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840c7bd0 con 0x7f28840b8360 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f2888fec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840c7e80 con 0x7f28840c7210 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f28841596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f28841596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60414/0 (socket says 172.21.15.111:60414) 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 -- 172.21.15.111:0/3680867321 learned_addr learned my addr 172.21.15.111:0/3680867321 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 -- 172.21.15.111:0/3680867321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840c7210 msgr2=0x7f28840c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840c7210 0x7f28840c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 -- 172.21.15.111:0/3680867321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28840b8360 msgr2=0x7f28840b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28840b8360 0x7f28840b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.547+0000 7f288259c640 1 -- 172.21.15.111:0/3680867321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840c8160 con 0x7f286c004d00 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.548+0000 7f288259c640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f28841596e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2874011e30 tx=0x7f287400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.548+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400e040 con 0x7f286c004d00 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.548+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400f040 con 0x7f286c004d00 2023-12-04T05:40:20.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.548+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28840c8440 con 0x7f286c004d00 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.548+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874009c00 con 0x7f286c004d00 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.548+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28840c8a00 con 0x7f286c004d00 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.549+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2884067460 con 0x7f286c004d00 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.549+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f2874015850 con 0x7f286c004d00 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.550+0000 7f2872ffd640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f286406b2a0 0x7f286406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.550+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f287408aa90 con 0x7f286c004d00 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.550+0000 7f2881d9b640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f286406b2a0 0x7f286406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:20.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.550+0000 7f2881d9b640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f286406b2a0 0x7f286406d760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28841f6300 tx=0x7f2878009320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:20.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.554+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2874055990 con 0x7f286c004d00 2023-12-04T05:40:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.755+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28840d39b0 con 0x7f286c004d00 2023-12-04T05:40:20.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.756+0000 7f2872ffd640 1 -- 172.21.15.111:0/3680867321 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2874058fb0 con 0x7f286c004d00 2023-12-04T05:40:20.757 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:20.758 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:20.758 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:20.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f286406b2a0 msgr2=0x7f286406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f286406b2a0 0x7f286406d760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28841f6300 tx=0x7f2878009320 comp rx=0 tx=0).stop 2023-12-04T05:40:20.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 msgr2=0x7f28841596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:20.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f28841596e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2874011e30 tx=0x7f287400f580 comp rx=0 tx=0).stop 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 shutdown_connections 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f286406b2a0 0x7f286406d760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f28840c7210 0x7f28840c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28840b8360 0x7f28840b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 --2- 172.21.15.111:0/3680867321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f286c004d00 0x7f28841596e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 >> 172.21.15.111:0/3680867321 conn(0x7f288409c460 msgr2=0x7f288415d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 shutdown_connections 2023-12-04T05:40:20.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:20.758+0000 7f2888fec640 1 -- 172.21.15.111:0/3680867321 wait complete. 2023-12-04T05:40:21.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-04T05:40:10.723946+0000', 'modified': '2023-12-04T05:40:13.542461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5235}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5235': {'gid': 5235, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6837/975894294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6835', 'nonce': 975894294}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 975894294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-04T05:40:21.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-04T05:40:21.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-04T05:40:21.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.274+0000 7ff280b97640 1 Processor -- start 2023-12-04T05:40:21.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 -- start start 2023-12-04T05:40:21.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff27c1544c0 0x7ff27c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff27c06d250 0x7ff27c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06ba10 0x7ff27c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c06bec0 con 0x7ff27c06d250 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c0ad5d0 con 0x7ff27c1544c0 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.275+0000 7ff280b97640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c06df90 con 0x7ff27c06ba10 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff27c06d250 0x7ff27c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff27c06d250 0x7ff27c06b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60430/0 (socket says 172.21.15.111:60430) 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 -- 172.21.15.111:0/1879526718 learned_addr learned my addr 172.21.15.111:0/1879526718 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff27ad9d640 1 --2- 172.21.15.111:0/1879526718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06ba10 0x7ff27c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff27a59c640 1 --2- 172.21.15.111:0/1879526718 >> v2:172.21.15.192:3300/0 conn(0x7ff27c1544c0 0x7ff27c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 -- 172.21.15.111:0/1879526718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06ba10 msgr2=0x7ff27c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 --2- 172.21.15.111:0/1879526718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06ba10 0x7ff27c160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 -- 172.21.15.111:0/1879526718 >> v2:172.21.15.192:3300/0 conn(0x7ff27c1544c0 msgr2=0x7ff27c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 --2- 172.21.15.111:0/1879526718 >> v2:172.21.15.192:3300/0 conn(0x7ff27c1544c0 0x7ff27c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.276+0000 7ff279d9b640 1 -- 172.21.15.111:0/1879526718 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c1548a0 con 0x7ff27c06d250 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff279d9b640 1 --2- 172.21.15.111:0/1879526718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06d250 0x7ff27c06b4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff27c06c070 tx=0x7ff26c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b19a7aaf77a32 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff27959a640 1 -- 172.21.15.111:0/1879526718 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c031870 con 0x7ff27c06d250 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff27959a640 1 -- 172.21.15.111:0/1879526718 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c0319d0 con 0x7ff27c06d250 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff27959a640 1 -- 172.21.15.111:0/1879526718 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c031d60 con 0x7ff27c06d250 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 -- 172.21.15.111:0/1879526718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06d250 msgr2=0x7ff27c06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 --2- 172.21.15.111:0/1879526718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06d250 0x7ff27c06b4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff27c06c070 tx=0x7ff26c02b910 comp rx=0 tx=0).stop 2023-12-04T05:40:21.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 -- 172.21.15.111:0/1879526718 shutdown_connections 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 --2- 172.21.15.111:0/1879526718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06ba10 0x7ff27c160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 --2- 172.21.15.111:0/1879526718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06d250 0x7ff27c06b4d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 --2- 172.21.15.111:0/1879526718 >> v2:172.21.15.192:3300/0 conn(0x7ff27c1544c0 0x7ff27c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 -- 172.21.15.111:0/1879526718 >> 172.21.15.111:0/1879526718 conn(0x7ff27c0b1b50 msgr2=0x7ff27c070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 -- 172.21.15.111:0/1879526718 shutdown_connections 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.277+0000 7ff280b97640 1 -- 172.21.15.111:0/1879526718 wait complete. 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 Processor -- start 2023-12-04T05:40:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 -- start start 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 0x7ff27c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06d250 0x7ff27c141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c1544c0 0x7ff27c1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1f7430 con 0x7ff27c06ba10 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c147d00 con 0x7ff27c1544c0 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.279+0000 7ff280b97640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1f6e70 con 0x7ff27c06d250 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 0x7ff27c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 0x7ff27c0b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60432/0 (socket says 172.21.15.111:60432) 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 -- 172.21.15.111:0/3902735242 learned_addr learned my addr 172.21.15.111:0/3902735242 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27ad9d640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c1544c0 0x7ff27c1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff279d9b640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06d250 0x7ff27c141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 -- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06d250 msgr2=0x7ff27c141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06d250 0x7ff27c141790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 -- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c1544c0 msgr2=0x7ff27c1f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c1544c0 0x7ff27c1f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 -- 172.21.15.111:0/3902735242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0094d0 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff279d9b640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06d250 0x7ff27c141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 0x7ff27c0b2580 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d5f0 tx=0x7ff27000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff26affd640 1 -- 172.21.15.111:0/3902735242 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700125f0 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff26affd640 1 -- 172.21.15.111:0/3902735242 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270012750 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c1f7150 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff26affd640 1 -- 172.21.15.111:0/3902735242 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270012b00 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.280+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c148050 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.281+0000 7ff26affd640 1 -- 172.21.15.111:0/3902735242 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff270012c60 con 0x7ff27c06ba10 2023-12-04T05:40:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.282+0000 7ff26affd640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff25c06b2a0 0x7ff25c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.282+0000 7ff26affd640 1 -- 172.21.15.111:0/3902735242 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff27008a010 con 0x7ff27c06ba10 2023-12-04T05:40:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.282+0000 7ff279d9b640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff25c06b2a0 0x7ff25c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.282+0000 7ff279d9b640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff25c06b2a0 0x7ff25c06d760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff26c036000 tx=0x7ff26c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.283+0000 7ff283806740 1 Processor -- start 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.283+0000 7ff283806740 1 -- start start 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06afa0 0x56110d03df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff283806740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d03e480 0x56110d06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff283806740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110cffed10 con 0x56110d06b380 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110cfb0b90 con 0x56110d06afa0 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110cfae280 con 0x56110d03e480 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60444/0 (socket says 172.21.15.111:60444) 2023-12-04T05:40:21.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 -- 172.21.15.111:0/2001754377 learned_addr learned my addr 172.21.15.111:0/2001754377 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff27ad9d640 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d03e480 0x56110d06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff27a59c640 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06afa0 0x56110d03df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 -- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d03e480 msgr2=0x56110d06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d03e480 0x56110d06e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 -- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06afa0 msgr2=0x56110d03df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06afa0 0x56110d03df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 -- 172.21.15.111:0/2001754377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110cf6f500 con 0x56110d06b380 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.284+0000 7ff279d9b640 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d06dd30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff26c044500 tx=0x7ff26c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a5a20e05d15a32 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff25bfff640 1 -- 172.21.15.111:0/2001754377 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c049070 con 0x56110d06b380 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff25bfff640 1 -- 172.21.15.111:0/2001754377 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c04e470 con 0x56110d06b380 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff25bfff640 1 -- 172.21.15.111:0/2001754377 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c04e800 con 0x56110d06b380 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 -- 172.21.15.111:0/2001754377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 msgr2=0x56110d06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d06dd30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff26c044500 tx=0x7ff26c030000 comp rx=0 tx=0).stop 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 -- 172.21.15.111:0/2001754377 shutdown_connections 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d03e480 0x56110d06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d06dd30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 --2- 172.21.15.111:0/2001754377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06afa0 0x56110d03df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 -- 172.21.15.111:0/2001754377 >> 172.21.15.111:0/2001754377 conn(0x56110d0401b0 msgr2=0x56110d088d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 -- 172.21.15.111:0/2001754377 shutdown_connections 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.285+0000 7ff283806740 1 -- 172.21.15.111:0/2001754377 wait complete. 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.286+0000 7ff283806740 1 -- start start 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.286+0000 7ff283806740 1 Processor -- start 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d05f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d0821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 0x56110d082730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110cfa9b70 con 0x56110d06b380 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110cf63e00 con 0x56110d08b560 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110cf63aa0 con 0x56110d06afa0 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff283806740 10 client.0 _ll_register_callbacks cb 0x56110cf682d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d0821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d0821f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60452/0 (socket says 172.21.15.111:60452) 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 -- 172.21.15.111:0/1405148333 learned_addr learned my addr 172.21.15.111:0/1405148333 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 -- 172.21.15.111:0/1405148333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 msgr2=0x56110d05f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d05f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 -- 172.21.15.111:0/1405148333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 msgr2=0x56110d082730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 0x56110d082730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 -- 172.21.15.111:0/1405148333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c1548a0 con 0x56110d06b380 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff279d9b640 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d0821f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff26c02bf20 tx=0x7ff26c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142c0e44e009c54a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.287+0000 7ff2597fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:40:21.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff2597fa640 1 -- 172.21.15.111:0/1405148333 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c049040 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff2597fa640 1 -- 172.21.15.111:0/1405148333 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff26c03f070 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56110cfa2b50 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff2597fa640 1 -- 172.21.15.111:0/1405148333 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c059680 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 10 client.5442 fetch_fsmap learned FSMap version 14 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x56110cfa81f0 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff2597fa640 1 -- 172.21.15.111:0/1405148333 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56110cfa2b50 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff2597fa640 1 -- 172.21.15.111:0/1405148333 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff26c055030 con 0x56110d06b380 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 10 client.5442 fetch_fsmap finished waiting for FSMap version 14 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 10 client.5442 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 20 client.5442 populate_metadata read hostname 'smithi111' 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d09c960 0x56110d070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 4 client.5442 mds_command: new command op to 5235 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 --> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56110cfa2870 con 0x56110d09c960 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff27a59c640 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d09c960 0x56110d070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff27a59c640 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d09c960 0x56110d070050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115259b8a5b7ac40 server_cookie=436e3e6a67cf9467 in_seq=0 out_seq=0 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.288+0000 7ff2597fa640 10 client.5442 ms_handle_connect on v2:172.21.15.111:6835/975894294 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff2597fa640 1 -- 172.21.15.111:0/1405148333 <== mds.0 v2:172.21.15.111:6835/975894294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56110cfa2870 con 0x56110d09c960 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff2597fa640 10 client.5442 handle_command_reply: tid=0 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 client.5442 shutdown 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 msgr2=0x56110d0821f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d0821f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff26c02bf20 tx=0x7ff26c004730 comp rx=0 tx=0).stop 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 shutdown_connections 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 0x56110d082730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d09c960 0x56110d070050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d0821f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 --2- 172.21.15.111:0/1405148333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d05f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff2597fa640 0 client.5442 ms_handle_reset on v2:172.21.15.111:6835/975894294 2023-12-04T05:40:21.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 >> 172.21.15.111:0/1405148333 conn(0x56110d03e480 msgr2=0x56110d08c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 shutdown_connections 2023-12-04T05:40:21.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 1 -- 172.21.15.111:0/1405148333 wait complete. 2023-12-04T05:40:21.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.289+0000 7ff283806740 20 client.5442 trim_cache size 0 max 16384 2023-12-04T05:40:21.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.305+0000 7ff283806740 1 Processor -- start 2023-12-04T05:40:21.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.305+0000 7ff283806740 1 -- start start 2023-12-04T05:40:21.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d041c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff283806740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d08c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 0x56110d08c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff283806740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110cf29270 con 0x56110d06b380 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110cf4e740 con 0x56110d08b560 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110d083970 con 0x56110d06afa0 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d08c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d08c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60468/0 (socket says 172.21.15.111:60468) 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 -- 172.21.15.111:0/713449089 learned_addr learned my addr 172.21.15.111:0/713449089 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 -- 172.21.15.111:0/713449089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 msgr2=0x56110d041c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d041c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 -- 172.21.15.111:0/713449089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 msgr2=0x56110d08c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 0x56110d08c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 -- 172.21.15.111:0/713449089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110cf45d00 con 0x56110d06b380 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff279d9b640 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d08c240 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff26c031960 tx=0x7ff26c054000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb13cc6ab556a504 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff268ff9640 1 -- 172.21.15.111:0/713449089 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c049020 con 0x56110d06b380 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff268ff9640 1 -- 172.21.15.111:0/713449089 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c059840 con 0x56110d06b380 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.306+0000 7ff268ff9640 1 -- 172.21.15.111:0/713449089 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c059c90 con 0x56110d06b380 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 -- 172.21.15.111:0/713449089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 msgr2=0x56110d08c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d08c240 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff26c031960 tx=0x7ff26c054000 comp rx=0 tx=0).stop 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 -- 172.21.15.111:0/713449089 shutdown_connections 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d08b560 0x56110d08c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d06b380 0x56110d08c240 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 --2- 172.21.15.111:0/713449089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d041c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 -- 172.21.15.111:0/713449089 >> 172.21.15.111:0/713449089 conn(0x56110d03e480 msgr2=0x56110d069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 -- 172.21.15.111:0/713449089 shutdown_connections 2023-12-04T05:40:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 -- 172.21.15.111:0/713449089 wait complete. 2023-12-04T05:40:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 -- start start 2023-12-04T05:40:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.307+0000 7ff283806740 1 Processor -- start 2023-12-04T05:40:21.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d097c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06b380 0x56110d0f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 0x56110d088bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110d05da30 con 0x56110d08b560 2023-12-04T05:40:21.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110d0a27c0 con 0x56110d06b380 2023-12-04T05:40:21.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110d0a25f0 con 0x56110d06afa0 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff283806740 10 client.0 _ll_register_callbacks cb 0x7ff26c04e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff27ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 0x56110d088bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff27ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 0x56110d088bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60478/0 (socket says 172.21.15.111:60478) 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.308+0000 7ff27ad9d640 1 -- 172.21.15.111:0/3166165460 learned_addr learned my addr 172.21.15.111:0/3166165460 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d097c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff279d9b640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06b380 0x56110d0f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27ad9d640 1 -- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 msgr2=0x56110d097c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27ad9d640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d097c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27ad9d640 1 -- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06b380 msgr2=0x56110d0f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27ad9d640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06b380 0x56110d0f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27ad9d640 1 -- 172.21.15.111:0/3166165460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56110cfa81f0 con 0x56110d08b560 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d097c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff27ad9d640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 0x56110d088bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff2740042f0 tx=0x7ff2740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b308c9e540f1c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff2597fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff2597fa640 1 -- 172.21.15.111:0/3166165460 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274009b80 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff2597fa640 1 -- 172.21.15.111:0/3166165460 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff274007980 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26c04e470 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff2597fa640 1 -- 172.21.15.111:0/3166165460 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff274007b80 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 10 client.5448 fetch_fsmap learned FSMap version 14 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x56110cf6b4f0 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff2597fa640 1 -- 172.21.15.111:0/3166165460 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c04e470 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff2597fa640 1 -- 172.21.15.111:0/3166165460 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff274014070 con 0x56110d08b560 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 10 client.5448 fetch_fsmap finished waiting for FSMap version 14 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 10 client.5448 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 20 client.5448 populate_metadata read hostname 'smithi111' 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d08c4d0 0x56110d042be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 4 client.5448 mds_command: new command op to 5235 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.309+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 --> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56110ccaa6c0 con 0x56110d08c4d0 2023-12-04T05:40:21.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d08c4d0 0x56110d042be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff2597fa640 10 client.5448 ms_handle_connect on v2:172.21.15.111:6835/975894294 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff27a59c640 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d08c4d0 0x56110d042be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136bf83b5d3c8946 server_cookie=cc8aeb02b66d9885 in_seq=0 out_seq=0 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff2597fa640 1 -- 172.21.15.111:0/3166165460 <== mds.0 v2:172.21.15.111:6835/975894294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56110ccaa6c0 con 0x56110d08c4d0 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff2597fa640 10 client.5448 handle_command_reply: tid=0 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 client.5448 shutdown 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 msgr2=0x56110d088bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 0x56110d088bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff2740042f0 tx=0x7ff2740047b0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 shutdown_connections 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:6835/975894294,v1:172.21.15.111:6837/975894294] conn(0x56110d08c4d0 0x56110d042be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56110d08b560 0x56110d088bf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff2597fa640 0 client.5448 ms_handle_reset on v2:172.21.15.111:6835/975894294 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56110d06b380 0x56110d0f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 --2- 172.21.15.111:0/3166165460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56110d06afa0 0x56110d097c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 >> 172.21.15.111:0/3166165460 conn(0x56110d03e480 msgr2=0x56110d041c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 shutdown_connections 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 1 -- 172.21.15.111:0/3166165460 wait complete. 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.310+0000 7ff283806740 20 client.5448 trim_cache size 0 max 16384 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "9686316e-a01b-4981-b0b7-e21bdb185d5e", 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5235, 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-12-04T05:40:21.318 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 9.9134844409999996, 2023-12-04T05:40:21.319 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 14, 2023-12-04T05:40:21.319 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 33, 2023-12-04T05:40:21.319 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 33, 2023-12-04T05:40:21.319 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 16.749949032 2023-12-04T05:40:21.319 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff25c06b2a0 msgr2=0x7ff25c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff25c06b2a0 0x7ff25c06d760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff26c036000 tx=0x7ff26c03c040 comp rx=0 tx=0).stop 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 msgr2=0x7ff27c0b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 0x7ff27c0b2580 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d5f0 tx=0x7ff27000dab0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 shutdown_connections 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff25c06b2a0 0x7ff25c06d760 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c1544c0 0x7ff27c1f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff27c06d250 0x7ff27c141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 --2- 172.21.15.111:0/3902735242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff27c06ba10 0x7ff27c0b2580 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 >> 172.21.15.111:0/3902735242 conn(0x7ff27c0b1b50 msgr2=0x7ff27c0ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 shutdown_connections 2023-12-04T05:40:21.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.311+0000 7ff280b97640 1 -- 172.21.15.111:0/3902735242 wait complete. 2023-12-04T05:40:21.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:40:21.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.420+0000 7faa5f7c5640 1 Processor -- start 2023-12-04T05:40:21.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.420+0000 7faa5f7c5640 1 -- start start 2023-12-04T05:40:21.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5f7c5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faa5813cee0 0x7faa5806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5f7c5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa58140200 0x7faa5806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5f7c5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa580aea20 con 0x7faa5815a490 2023-12-04T05:40:21.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5f7c5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5813d2c0 con 0x7faa58140200 2023-12-04T05:40:21.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5f7c5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5815b3a0 con 0x7faa5813cee0 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa58140200 0x7faa5806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faa58140200 0x7faa5806c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60492/0 (socket says 172.21.15.111:60492) 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 -- 172.21.15.111:0/2209883127 learned_addr learned my addr 172.21.15.111:0/2209883127 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5d561640 1 --2- 172.21.15.111:0/2209883127 >> v2:172.21.15.192:3300/0 conn(0x7faa5813cee0 0x7faa5806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 -- 172.21.15.111:0/2209883127 >> v2:172.21.15.192:3300/0 conn(0x7faa5813cee0 msgr2=0x7faa5806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 --2- 172.21.15.111:0/2209883127 >> v2:172.21.15.192:3300/0 conn(0x7faa5813cee0 0x7faa5806c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 -- 172.21.15.111:0/2209883127 >> v1:172.21.15.192:6789/0 conn(0x7faa5815a490 legacy=0x7faa5815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.421+0000 7faa5cd60640 1 -- 172.21.15.111:0/2209883127 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5815b0c0 con 0x7faa58140200 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5cd60640 1 --2- 172.21.15.111:0/2209883127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa58140200 0x7faa5806c790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faa580b2280 tx=0x7faa4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2acb496a68c1a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa4ffff640 1 -- 172.21.15.111:0/2209883127 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48031870 con 0x7faa58140200 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa4ffff640 1 -- 172.21.15.111:0/2209883127 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480319d0 con 0x7faa58140200 2023-12-04T05:40:21.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa4ffff640 1 -- 172.21.15.111:0/2209883127 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48031d60 con 0x7faa58140200 2023-12-04T05:40:21.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5f7c5640 1 -- 172.21.15.111:0/2209883127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa58140200 msgr2=0x7faa5806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/2209883127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa58140200 0x7faa5806c790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faa580b2280 tx=0x7faa4802b910 comp rx=0 tx=0).stop 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5f7c5640 1 -- 172.21.15.111:0/2209883127 shutdown_connections 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/2209883127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa58140200 0x7faa5806c790 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/2209883127 >> v2:172.21.15.192:3300/0 conn(0x7faa5813cee0 0x7faa5806c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.422+0000 7faa5f7c5640 1 -- 172.21.15.111:0/2209883127 >> 172.21.15.111:0/2209883127 conn(0x7faa5806a4c0 msgr2=0x7faa5806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.423+0000 7faa5f7c5640 1 -- 172.21.15.111:0/2209883127 shutdown_connections 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.423+0000 7faa5f7c5640 1 -- 172.21.15.111:0/2209883127 wait complete. 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 Processor -- start 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 -- start start 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa5813cee0 0x7faa58071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58140200 0x7faa58071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 0x7faa581f0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5814e370 con 0x7faa5815a490 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58071a70 con 0x7faa58140200 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5f7c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5814e5c0 con 0x7faa5813cee0 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.424+0000 7faa5dd62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 0x7faa581f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 0x7faa581f0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60500/0 (socket says 172.21.15.111:60500) 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 -- 172.21.15.111:0/4034142497 learned_addr learned my addr 172.21.15.111:0/4034142497 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5d561640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa5813cee0 0x7faa58071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5cd60640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58140200 0x7faa58071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 -- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa5813cee0 msgr2=0x7faa58071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa5813cee0 0x7faa58071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 -- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58140200 msgr2=0x7faa58071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58140200 0x7faa58071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 -- 172.21.15.111:0/4034142497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480094d0 con 0x7faa5815a490 2023-12-04T05:40:21.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5cd60640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58140200 0x7faa58071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:21.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.425+0000 7faa5dd62640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 0x7faa581f0970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faa5000c6e0 tx=0x7faa5000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.427+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50003b70 con 0x7faa5815a490 2023-12-04T05:40:21.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.427+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa50003cf0 con 0x7faa5815a490 2023-12-04T05:40:21.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.427+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5814e8a0 con 0x7faa5815a490 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.427+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50005310 con 0x7faa5815a490 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.427+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5814ee60 con 0x7faa5815a490 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.428+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5814f050 con 0x7faa5815a490 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.428+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7faa50005470 con 0x7faa5815a490 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.429+0000 7faa4dffb640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa3c06b2a0 0x7faa3c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.429+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa50089800 con 0x7faa5815a490 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.429+0000 7faa5d561640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa3c06b2a0 0x7faa3c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:21.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.429+0000 7faa5d561640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa3c06b2a0 0x7faa3c06d760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa54006fd0 tx=0x7faa54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:21.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.433+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa50054700 con 0x7faa5815a490 2023-12-04T05:40:21.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.636+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa580693d0 con 0x7faa5815a490 2023-12-04T05:40:21.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.636+0000 7faa4dffb640 1 -- 172.21.15.111:0/4034142497 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa50003e60 con 0x7faa5815a490 2023-12-04T05:40:21.638 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa3c06b2a0 msgr2=0x7faa3c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa3c06b2a0 0x7faa3c06d760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa54006fd0 tx=0x7faa54008040 comp rx=0 tx=0).stop 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 msgr2=0x7faa581f0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 0x7faa581f0970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faa5000c6e0 tx=0x7faa5000cba0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 shutdown_connections 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faa3c06b2a0 0x7faa3c06d760 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa5815a490 0x7faa581f0970 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58140200 0x7faa58071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 --2- 172.21.15.111:0/4034142497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa5813cee0 0x7faa58071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.638+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 >> 172.21.15.111:0/4034142497 conn(0x7faa5806a4c0 msgr2=0x7faa58150e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.639+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 shutdown_connections 2023-12-04T05:40:21.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:21.639+0000 7faa5f7c5640 1 -- 172.21.15.111:0/4034142497 wait complete. 2023-12-04T05:40:21.650 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:40:21.651 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:40:21.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:21.651 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-12-04T05:40:21.667 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42096sec preferred_lft 42096sec 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4e:91:07:f7:96:4c brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:40:21.668 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::142c:d5ff:fef0:ffc5/64 scope link 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8e:82:5e:fe:4f:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::8c82:5eff:fefe:4fbf/64 scope link 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4e:91:07:f7:96:4c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::4c91:7ff:fef7:964c/64 scope link 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:21.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:21.670 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:40:21.733 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-12-04T05:40:21.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:21.734 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-12-04T05:40:21.789 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-12-04T05:40:21.790 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:40:21.790 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:40:21.790 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:40:21.790 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:40:21.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:21.790 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:40:21.790 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:40:21.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:40:21.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:40:21.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:40:21.791 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:40:21.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:21.885 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:40:21.899 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-04T05:40:21.900 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-04T05:40:21.900 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:40:21.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:21.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-12-04T05:40:21.967 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-04T05:40:21.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:40:21.968 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:22.022 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:40:22.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:40:22.022 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:22.077 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:40:22.144 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:40:22.200 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:40:22.201 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:22.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:22.203 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:40:22.255 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:40:22.255 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-04T05:40:22.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:40:22.324 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:40:22.381 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:40:22.382 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:22.383 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:22.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:22.384 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:40:22.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2023-12-04T05:40:22.382+0000 7f27e0a3f480 -1 init, newargv = 0x55ed9ebde1c0 newargc=15 2023-12-04T05:40:22.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[103842]: starting ceph client 2023-12-04T05:40:22.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[103842]: starting fuse 2023-12-04T05:40:22.399 INFO:teuthology.orchestra.run.smithi111.stdout:41 2023-12-04T05:40:22.399 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:40:22.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:40:22.399 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:22.400 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2023-12-04T05:40:22.401 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-12-04T05:40:22.519 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.103842.asok 2023-12-04T05:40:22.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:40:22.529 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103842.asok 2023-12-04T05:40:22.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:22.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103842.asok status 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-04T05:40:22.664 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "103842", 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5460, 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 5460 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2393531939 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:22.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2393531939 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.5460 192.168.0.1:0/2393531939", 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/2393531939", 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 14, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 33, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-12-04T05:40:22.666 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-12-04T05:40:22.667 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:40:22.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:22.673 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-12-04T05:40:22.673 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:22.687 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-12-04T05:40:22.688 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:22.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:22.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:22.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:40:22.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.895+0000 7f0c33207640 1 Processor -- start 2023-12-04T05:40:22.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.895+0000 7f0c33207640 1 -- start start 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c33207640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0c2c06cae0 0x7f0c2c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c33207640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c0b3310 0x7f0c2c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c33207640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c154220 con 0x7f0c2c15dab0 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c33207640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c15d2f0 con 0x7f0c2c06cae0 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c33207640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c06d400 con 0x7f0c2c0b3310 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c30fa3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0c2c06cae0 0x7f0c2c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c317a4640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0c2c15dab0 0x7f0c2c0703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58204/0 (socket says 172.21.15.111:58204) 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c317a4640 1 -- 172.21.15.111:0/816856246 learned_addr learned my addr 172.21.15.111:0/816856246 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c23fff640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c0b3310 0x7f0c2c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c30fa3640 1 -- 172.21.15.111:0/816856246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c0b3310 msgr2=0x7f0c2c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c30fa3640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c0b3310 0x7f0c2c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c30fa3640 1 -- 172.21.15.111:0/816856246 >> v1:172.21.15.111:6789/0 conn(0x7f0c2c15dab0 legacy=0x7f0c2c0703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:22.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c237fe640 1 -- 172.21.15.111:0/816856246 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683277460 0 0) 0x7f0c24001490 con 0x7f0c2c15dab0 2023-12-04T05:40:22.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c30fa3640 1 -- 172.21.15.111:0/816856246 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c15a5c0 con 0x7f0c2c06cae0 2023-12-04T05:40:22.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.896+0000 7f0c23fff640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c0b3310 0x7f0c2c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:22.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c30fa3640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c06cec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800a940 tx=0x7f0c2802cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7207ad26e46189 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c237fe640 1 -- 172.21.15.111:0/816856246 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2802db30 con 0x7f0c2c06cae0 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c237fe640 1 -- 172.21.15.111:0/816856246 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2802dc90 con 0x7f0c2c06cae0 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c237fe640 1 -- 172.21.15.111:0/816856246 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28035440 con 0x7f0c2c06cae0 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 -- 172.21.15.111:0/816856246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c06cae0 msgr2=0x7f0c2c06cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c06cec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800a940 tx=0x7f0c2802cd80 comp rx=0 tx=0).stop 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 -- 172.21.15.111:0/816856246 shutdown_connections 2023-12-04T05:40:22.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c0b3310 0x7f0c2c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:22.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 --2- 172.21.15.111:0/816856246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c06cec0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:22.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 -- 172.21.15.111:0/816856246 >> 172.21.15.111:0/816856246 conn(0x7f0c2c06a4c0 msgr2=0x7f0c2c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:22.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 -- 172.21.15.111:0/816856246 shutdown_connections 2023-12-04T05:40:22.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.897+0000 7f0c33207640 1 -- 172.21.15.111:0/816856246 wait complete. 2023-12-04T05:40:22.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.898+0000 7f0c33207640 1 Processor -- start 2023-12-04T05:40:22.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 -- start start 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 0x7f0c2c1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c15dab0 0x7f0c2c141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c140df0 con 0x7f0c2c0b3310 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1415a0 con 0x7f0c2c06cae0 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.899+0000 7f0c33207640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c141870 con 0x7f0c2c15dab0 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 0x7f0c2c1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 0x7f0c2c1408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60522/0 (socket says 172.21.15.111:60522) 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 -- 172.21.15.111:0/312260071 learned_addr learned my addr 172.21.15.111:0/312260071 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:22.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c317a4640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c15dab0 0x7f0c2c141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c30fa3640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 -- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c15dab0 msgr2=0x7f0c2c141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c15dab0 0x7f0c2c141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 -- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c2c06cae0 msgr2=0x7f0c2c143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 -- 172.21.15.111:0/312260071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2800a5a0 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c30fa3640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c23fff640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 0x7f0c2c1408b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c7f0 tx=0x7f0c1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c007a30 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c007b90 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c141b50 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00ce40 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.900+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c142110 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.901+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c0693d0 con 0x7f0c2c0b3310 2023-12-04T05:40:22.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.902+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0c1c0113f0 con 0x7f0c2c0b3310 2023-12-04T05:40:22.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.902+0000 7f0c217fa640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0c1806b2a0 0x7f0c1806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:22.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.902+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c1c089cf0 con 0x7f0c2c0b3310 2023-12-04T05:40:22.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.902+0000 7f0c30fa3640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0c1806b2a0 0x7f0c1806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:22.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.903+0000 7f0c30fa3640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0c1806b2a0 0x7f0c1806d760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c28037000 tx=0x7f0c28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:22.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:22.906+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c1c08e070 con 0x7f0c2c0b3310 2023-12-04T05:40:23.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.110+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0c2c1576b0 con 0x7f0c2c0b3310 2023-12-04T05:40:23.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.110+0000 7f0c217fa640 1 -- 172.21.15.111:0/312260071 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0c1c01f070 con 0x7f0c2c0b3310 2023-12-04T05:40:23.112 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:40:23.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0c1806b2a0 msgr2=0x7f0c1806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:23.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0c1806b2a0 0x7f0c1806d760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0c28037000 tx=0x7f0c28008040 comp rx=0 tx=0).stop 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 msgr2=0x7f0c2c1408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 0x7f0c2c1408b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c7f0 tx=0x7f0c1c00cbf0 comp rx=0 tx=0).stop 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 shutdown_connections 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0c1806b2a0 0x7f0c1806d760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c2c15dab0 0x7f0c2c141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c2c0b3310 0x7f0c2c1408b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 --2- 172.21.15.111:0/312260071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c2c06cae0 0x7f0c2c143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 >> 172.21.15.111:0/312260071 conn(0x7f0c2c06a4c0 msgr2=0x7f0c2c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 shutdown_connections 2023-12-04T05:40:23.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:23.112+0000 7f0c33207640 1 -- 172.21.15.111:0/312260071 wait complete. 2023-12-04T05:40:23.124 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:40:23.124 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:40:23.124 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-04T05:40:23.124 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-04T05:40:23.125 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:23.125 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:40:23.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:40:23.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:40:23.125 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:40:23.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.125 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-04T05:40:23.144 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:23.145 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42083sec preferred_lft 42083sec 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 1a:e1:a2:48:a2:71 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::495:82ff:fe57:3769/64 scope link 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:40:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 1a:e1:a2:48:a2:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::18e1:a2ff:fe48:a271/64 scope link 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether fe:d5:d3:8d:28:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::fcd5:d3ff:fe8d:28fa/64 scope link 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:23.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.147 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:40:23.216 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-04T05:40:23.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.218 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-04T05:40:23.276 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-04T05:40:23.276 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:40:23.276 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:40:23.276 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:40:23.276 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:40:23.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.276 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:40:23.276 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:40:23.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:40:23.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:40:23.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:40:23.277 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:40:23.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.376 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-04T05:40:23.392 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-04T05:40:23.392 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-04T05:40:23.392 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:40:23.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-04T05:40:23.460 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-04T05:40:23.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:40:23.460 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:23.515 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:40:23.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:40:23.516 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:23.571 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:40:23.639 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:40:23.695 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:40:23.696 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:40:23.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.697 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:40:23.750 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:40:23.750 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-04T05:40:23.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:40:23.819 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:40:23.875 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:40:23.876 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:40:23.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:23.878 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:40:23.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-04T05:40:23.876+0000 7f6f4cc4b480 -1 init, newargv = 0x55bf54951640 newargc=15 2023-12-04T05:40:23.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[96468]: starting ceph client 2023-12-04T05:40:23.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[96468]: starting fuse 2023-12-04T05:40:23.892 INFO:teuthology.orchestra.run.smithi192.stdout:41 2023-12-04T05:40:23.893 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-04T05:40:23.893 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:40:23.894 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:40:23.895 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-04T05:40:23.895 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-04T05:40:24.012 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.96468.asok 2023-12-04T05:40:24.015 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:40:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:40:24.022 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.96468.asok 2023-12-04T05:40:24.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:24.022 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.96468.asok status 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "96468", 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:40:24.158 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5170, 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5170 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3041332419 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:40:24.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3041332419 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5170 192.168.0.1:0/3041332419", 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/3041332419", 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 14, 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 33, 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-04T05:40:24.160 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-04T05:40:24.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:24.166 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-04T05:40:24.166 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:24.181 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-04T05:40:24.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:24.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:24.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:24.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2023-12-04T05:40:24.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 Processor -- start 2023-12-04T05:40:24.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 -- start start 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 0x7f47f80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f47f80d0010 0x7f47f8160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f815ddc0 con 0x7f47f815d480 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80d04a0 con 0x7f47f80d0010 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.361+0000 7f47fcbb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f815e0a0 con 0x7f47f815d860 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f659c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f47f815d480 0x7f47f80d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58210/0 (socket says 172.21.15.111:58210) 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f659c640 1 -- 172.21.15.111:0/3664917260 learned_addr learned my addr 172.21.15.111:0/3664917260 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f6d9d640 1 --2- 172.21.15.111:0/3664917260 >> v2:172.21.15.192:3300/0 conn(0x7f47f80d0010 0x7f47f8160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47effff640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 0x7f47f80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180381417 0 0) 0x7f47f815ddc0 con 0x7f47f815d480 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e4003520 con 0x7f47f815d480 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 793510237 0 0) 0x7f47e4003520 con 0x7f47f815d480 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 msgr2=0x7f47f80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 0x7f47f80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 >> v2:172.21.15.192:3300/0 conn(0x7f47f80d0010 msgr2=0x7f47f8160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 --2- 172.21.15.111:0/3664917260 >> v2:172.21.15.192:3300/0 conn(0x7f47f80d0010 0x7f47f8160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f815e380 con 0x7f47f815d480 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f47e8001670 con 0x7f47f815d480 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 >> v1:172.21.15.111:6789/0 conn(0x7f47f815d480 legacy=0x7f47f80d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 0x7f47e4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e4004d00 0x7f47e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d04a0 con 0x7f47e4003ef0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f815e0a0 con 0x7f47e4004d00 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e400b980 con 0x7f47f815d860 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47effff640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f659c640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 0x7f47e4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.362+0000 7f47f6d9d640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e4004d00 0x7f47e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47effff640 1 -- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 msgr2=0x7f47e4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47effff640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f815d860 0x7f47e4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47effff640 1 -- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e4004d00 msgr2=0x7f47e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47effff640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e4004d00 0x7f47e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47effff640 1 -- 172.21.15.111:0/3664917260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4003520 con 0x7f47e4003ef0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47effff640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47e40047c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009ac0 tx=0x7f47e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4345d76d2f6e581c server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.363+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e002c6b0 con 0x7f47e4003ef0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e002c810 con 0x7f47e4003ef0 2023-12-04T05:40:24.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47f5d9b640 1 -- 172.21.15.111:0/3664917260 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e002cba0 con 0x7f47e4003ef0 2023-12-04T05:40:24.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47fcbb0640 1 -- 172.21.15.111:0/3664917260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 msgr2=0x7f47e40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/3664917260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47e40047c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009ac0 tx=0x7f47e40057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47f6d9d640 1 -- 172.21.15.111:0/3664917260 reap_dead start 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47fcbb0640 1 -- 172.21.15.111:0/3664917260 shutdown_connections 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47fcbb0640 1 -- 172.21.15.111:0/3664917260 >> 172.21.15.111:0/3664917260 conn(0x7f47f809c540 msgr2=0x7f47f80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47fcbb0640 1 -- 172.21.15.111:0/3664917260 shutdown_connections 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.364+0000 7f47fcbb0640 1 -- 172.21.15.111:0/3664917260 wait complete. 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.365+0000 7f47fcbb0640 1 Processor -- start 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 -- start start 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47f80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f80b8830 0x7f47f8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f81574c0 0x7f47f81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8157970 con 0x7f47e4003ef0 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80b7790 con 0x7f47f81574c0 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47fcbb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80b7a70 con 0x7f47f80b8830 2023-12-04T05:40:24.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47f659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47f80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47f659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47f80b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60536/0 (socket says 172.21.15.111:60536) 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47f659c640 1 -- 172.21.15.111:0/2180975343 learned_addr learned my addr 172.21.15.111:0/2180975343 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47f6d9d640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f81574c0 0x7f47f81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.366+0000 7f47effff640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f80b8830 0x7f47f8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f659c640 1 -- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f80b8830 msgr2=0x7f47f8156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f659c640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f80b8830 0x7f47f8156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f659c640 1 -- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f81574c0 msgr2=0x7f47f81594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f659c640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f81574c0 0x7f47f81594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f659c640 1 -- 172.21.15.111:0/2180975343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80b7d50 con 0x7f47e4003ef0 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f6d9d640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f81574c0 0x7f47f81594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47effff640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f80b8830 0x7f47f8156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47f659c640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47f80b7090 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47e8007340 tx=0x7f47e8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800f940 con 0x7f47e4003ef0 2023-12-04T05:40:24.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800faa0 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800fe30 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80b8030 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.367+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f8159a30 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.368+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f8067980 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.368+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f47e801d460 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.369+0000 7f47eeffd640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47d806b2d0 0x7f47d806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.369+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47e8089e50 con 0x7f47e4003ef0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.369+0000 7f47effff640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47d806b2d0 0x7f47d806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.369+0000 7f47effff640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47d806b2d0 0x7f47d806d790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47e0036000 tx=0x7f47e0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:24.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.373+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47e8054cc0 con 0x7f47e4003ef0 2023-12-04T05:40:24.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.544+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) v1 -- 0x7f47f80b8550 con 0x7f47e4003ef0 2023-12-04T05:40:24.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.547+0000 7f47eeffd640 1 -- 172.21.15.111:0/2180975343 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v11)=0 v11) v1 ==== 121+0+0 (secure 0 0 0) 0x7f47f80b8550 con 0x7f47e4003ef0 2023-12-04T05:40:24.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47d806b2d0 msgr2=0x7f47d806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47d806b2d0 0x7f47d806d790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47e0036000 tx=0x7f47e0030000 comp rx=0 tx=0).stop 2023-12-04T05:40:24.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 msgr2=0x7f47f80b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47f80b7090 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47e8007340 tx=0x7f47e8007800 comp rx=0 tx=0).stop 2023-12-04T05:40:24.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 shutdown_connections 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47d806b2d0 0x7f47d806d790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47f81574c0 0x7f47f81594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47f80b8830 0x7f47f8156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 --2- 172.21.15.111:0/2180975343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e4003ef0 0x7f47f80b7090 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 >> 172.21.15.111:0/2180975343 conn(0x7f47f809c540 msgr2=0x7f47f815ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 shutdown_connections 2023-12-04T05:40:24.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.549+0000 7f47fcbb0640 1 -- 172.21.15.111:0/2180975343 wait complete. 2023-12-04T05:40:24.561 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:24.561 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:24.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.713+0000 7fd49f831640 1 Processor -- start 2023-12-04T05:40:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.714+0000 7fd49f831640 1 -- start start 2023-12-04T05:40:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.714+0000 7fd49f831640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4980b20a0 0x7fd4980d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.714+0000 7fd49f831640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.714+0000 7fd49f831640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4980cdcb0 con 0x7fd4980d1780 2023-12-04T05:40:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.714+0000 7fd49f831640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4980d39e0 con 0x7fd4980b20a0 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.714+0000 7fd49f831640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980d3cc0 con 0x7fd4980b2ad0 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49cdcc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4980b20a0 0x7fd4980d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49ddce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49d5cd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd4980d1780 0x7fd4980d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:41566/0 (socket says 172.21.15.111:41566) 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49d5cd640 1 -- 172.21.15.111:0/685813743 learned_addr learned my addr 172.21.15.111:0/685813743 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49ddce640 1 -- 172.21.15.111:0/685813743 >> v2:172.21.15.192:3300/0 conn(0x7fd4980b20a0 msgr2=0x7fd4980d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49ddce640 1 --2- 172.21.15.111:0/685813743 >> v2:172.21.15.192:3300/0 conn(0x7fd4980b20a0 0x7fd4980d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49ddce640 1 -- 172.21.15.111:0/685813743 >> v1:172.21.15.192:6789/0 conn(0x7fd4980d1780 legacy=0x7fd4980d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49ddce640 1 -- 172.21.15.111:0/685813743 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4980cdf50 con 0x7fd4980b2ad0 2023-12-04T05:40:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.715+0000 7fd49cdcc640 1 --2- 172.21.15.111:0/685813743 >> v2:172.21.15.192:3300/0 conn(0x7fd4980b20a0 0x7fd4980d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.716+0000 7fd49ddce640 1 --2- 172.21.15.111:0/685813743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980ce750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd490009d90 tx=0x7fd49002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42fa3cede9847073 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.716+0000 7fd48ffff640 1 -- 172.21.15.111:0/685813743 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49002cbc0 con 0x7fd4980b2ad0 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.716+0000 7fd48ffff640 1 -- 172.21.15.111:0/685813743 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49002cd20 con 0x7fd4980b2ad0 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.716+0000 7fd48ffff640 1 -- 172.21.15.111:0/685813743 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490034660 con 0x7fd4980b2ad0 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.716+0000 7fd49f831640 1 -- 172.21.15.111:0/685813743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 msgr2=0x7fd4980ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.716+0000 7fd49f831640 1 --2- 172.21.15.111:0/685813743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980ce750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd490009d90 tx=0x7fd49002be10 comp rx=0 tx=0).stop 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.717+0000 7fd49f831640 1 -- 172.21.15.111:0/685813743 shutdown_connections 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.717+0000 7fd49f831640 1 --2- 172.21.15.111:0/685813743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980ce750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.717+0000 7fd49f831640 1 --2- 172.21.15.111:0/685813743 >> v2:172.21.15.192:3300/0 conn(0x7fd4980b20a0 0x7fd4980d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.717+0000 7fd49f831640 1 -- 172.21.15.111:0/685813743 >> 172.21.15.111:0/685813743 conn(0x7fd49809c4e0 msgr2=0x7fd4980d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.717+0000 7fd49f831640 1 -- 172.21.15.111:0/685813743 shutdown_connections 2023-12-04T05:40:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.717+0000 7fd49f831640 1 -- 172.21.15.111:0/685813743 wait complete. 2023-12-04T05:40:24.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 Processor -- start 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 -- start start 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4980b20a0 0x7fd4981609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4980d1780 0x7fd4981578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980a0f70 con 0x7fd4980b20a0 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980a0cd0 con 0x7fd4980d1780 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.718+0000 7fd49f831640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b6f40 con 0x7fd4980b2ad0 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49d5cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4980b20a0 0x7fd4981609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4980d1780 0x7fd4981578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49d5cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4980b20a0 0x7fd4981609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60548/0 (socket says 172.21.15.111:60548) 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49cdcc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49d5cd640 1 -- 172.21.15.111:0/1528288489 learned_addr learned my addr 172.21.15.111:0/1528288489 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:24.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 -- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 msgr2=0x7fd4980a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 -- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4980b20a0 msgr2=0x7fd4981609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4980b20a0 0x7fd4981609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 -- 172.21.15.111:0/1528288489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488009400 con 0x7fd4980d1780 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49cdcc640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49ddce640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4980d1780 0x7fd4981578a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd490033000 tx=0x7fd490004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490040650 con 0x7fd4980d1780 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49002ce90 con 0x7fd4980d1780 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd490009a70 con 0x7fd4980d1780 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900409f0 con 0x7fd4980d1780 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.719+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4980b7500 con 0x7fd4980d1780 2023-12-04T05:40:24.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.720+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4980693d0 con 0x7fd4980d1780 2023-12-04T05:40:24.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.721+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd490040b50 con 0x7fd4980d1780 2023-12-04T05:40:24.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.721+0000 7fd48dffb640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd47c06b2a0 0x7fd47c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:24.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.721+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4900ac560 con 0x7fd4980d1780 2023-12-04T05:40:24.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.721+0000 7fd49d5cd640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd47c06b2a0 0x7fd47c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:24.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.722+0000 7fd49d5cd640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd47c06b2a0 0x7fd47c06d760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd494005880 tx=0x7fd494009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:24.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.726+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4900774f0 con 0x7fd4980d1780 2023-12-04T05:40:24.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.891+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4980d40b0 con 0x7fd4980d1780 2023-12-04T05:40:24.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.893+0000 7fd48dffb640 1 -- 172.21.15.111:0/1528288489 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fd490009450 con 0x7fd4980d1780 2023-12-04T05:40:24.895 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:24.895 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:24.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd47c06b2a0 msgr2=0x7fd47c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd47c06b2a0 0x7fd47c06d760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd494005880 tx=0x7fd494009040 comp rx=0 tx=0).stop 2023-12-04T05:40:24.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4980d1780 msgr2=0x7fd4981578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4980d1780 0x7fd4981578a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd490033000 tx=0x7fd490004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 shutdown_connections 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd47c06b2a0 0x7fd47c06d760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4980d1780 0x7fd4981578a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980b2ad0 0x7fd4980a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 --2- 172.21.15.111:0/1528288489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4980b20a0 0x7fd4981609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 >> 172.21.15.111:0/1528288489 conn(0x7fd49809c4e0 msgr2=0x7fd4980d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 shutdown_connections 2023-12-04T05:40:24.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:24.895+0000 7fd49f831640 1 -- 172.21.15.111:0/1528288489 wait complete. 2023-12-04T05:40:24.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:24.907 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:25.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 Processor -- start 2023-12-04T05:40:25.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 -- start start 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d78070c10 0x7f0d780ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d780cd020 0x7f0d780b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d78159c20 con 0x7f0d78070c10 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d780ced70 con 0x7f0d780cd020 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.010+0000 7f0d7dc32640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d780b50e0 con 0x7f0d78070ff0 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d78070c10 0x7f0d780ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d78070c10 0x7f0d780ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60558/0 (socket says 172.21.15.111:60558) 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 -- 172.21.15.111:0/471283011 learned_addr learned my addr 172.21.15.111:0/471283011 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d76ffd640 1 --2- 172.21.15.111:0/471283011 >> v2:172.21.15.192:3300/0 conn(0x7f0d780cd020 0x7f0d780b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d77fff640 1 --2- 172.21.15.111:0/471283011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 -- 172.21.15.111:0/471283011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 msgr2=0x7f0d78071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:25.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 --2- 172.21.15.111:0/471283011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 -- 172.21.15.111:0/471283011 >> v2:172.21.15.192:3300/0 conn(0x7f0d780cd020 msgr2=0x7f0d780b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 --2- 172.21.15.111:0/471283011 >> v2:172.21.15.192:3300/0 conn(0x7f0d780cd020 0x7f0d780b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.011+0000 7f0d777fe640 1 -- 172.21.15.111:0/471283011 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780cd790 con 0x7f0d78070c10 2023-12-04T05:40:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d777fe640 1 --2- 172.21.15.111:0/471283011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d780ce830 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d78067310 tx=0x7f0d6802bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8bab999932cae server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d767fc640 1 -- 172.21.15.111:0/471283011 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68004230 con 0x7f0d78070c10 2023-12-04T05:40:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d767fc640 1 -- 172.21.15.111:0/471283011 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d68004390 con 0x7f0d78070c10 2023-12-04T05:40:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d767fc640 1 -- 172.21.15.111:0/471283011 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6802c850 con 0x7f0d78070c10 2023-12-04T05:40:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 -- 172.21.15.111:0/471283011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 msgr2=0x7f0d780ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/471283011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d780ce830 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d78067310 tx=0x7f0d6802bcb0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 -- 172.21.15.111:0/471283011 shutdown_connections 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/471283011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/471283011 >> v2:172.21.15.192:3300/0 conn(0x7f0d780cd020 0x7f0d780b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/471283011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d780ce830 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 -- 172.21.15.111:0/471283011 >> 172.21.15.111:0/471283011 conn(0x7f0d780b0cf0 msgr2=0x7f0d780b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 -- 172.21.15.111:0/471283011 shutdown_connections 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.012+0000 7f0d7dc32640 1 -- 172.21.15.111:0/471283011 wait complete. 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.013+0000 7f0d7dc32640 1 Processor -- start 2023-12-04T05:40:25.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 -- start start 2023-12-04T05:40:25.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d781562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d780cd020 0x7f0d780af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d781ee640 con 0x7f0d78070c10 2023-12-04T05:40:25.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78142f60 con 0x7f0d780cd020 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d7dc32640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78157470 con 0x7f0d78070ff0 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d781562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d781562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60560/0 (socket says 172.21.15.111:60560) 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.014+0000 7f0d777fe640 1 -- 172.21.15.111:0/3899072603 learned_addr learned my addr 172.21.15.111:0/3899072603 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d77fff640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d780cd020 0x7f0d780af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d76ffd640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d777fe640 1 -- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 msgr2=0x7f0d78142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d777fe640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78142a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d777fe640 1 -- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d780cd020 msgr2=0x7f0d780af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d777fe640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d780cd020 0x7f0d780af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d777fe640 1 -- 172.21.15.111:0/3899072603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6800a580 con 0x7f0d78070c10 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d777fe640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d781562e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0d68036040 tx=0x7f0d680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003810 con 0x7f0d78070c10 2023-12-04T05:40:25.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d68003970 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d781568f0 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003d20 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.015+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d78156c80 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.016+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d781ee7b0 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.016+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0d68040470 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.017+0000 7f0d5ffff640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0d6406b2a0 0x7f0d6406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.017+0000 7f0d76ffd640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0d6406b2a0 0x7f0d6406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.017+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d680ac330 con 0x7f0d78070c10 2023-12-04T05:40:25.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.017+0000 7f0d76ffd640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0d6406b2a0 0x7f0d6406d760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d78156820 tx=0x7f0d6c00b3a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.021+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d781ee7b0 con 0x7f0d78070c10 2023-12-04T05:40:25.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.187+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d781ee780 con 0x7f0d78070c10 2023-12-04T05:40:25.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.187+0000 7f0d5ffff640 1 -- 172.21.15.111:0/3899072603 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d68077260 con 0x7f0d78070c10 2023-12-04T05:40:25.189 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0d6406b2a0 msgr2=0x7f0d6406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0d6406b2a0 0x7f0d6406d760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d78156820 tx=0x7f0d6c00b3a0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 msgr2=0x7f0d781562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d781562e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0d68036040 tx=0x7f0d680046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 shutdown_connections 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0d6406b2a0 0x7f0d6406d760 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d780cd020 0x7f0d780af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d78070ff0 0x7f0d78142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 --2- 172.21.15.111:0/3899072603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d78070c10 0x7f0d781562e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 >> 172.21.15.111:0/3899072603 conn(0x7f0d780b0cf0 msgr2=0x7f0d780b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 shutdown_connections 2023-12-04T05:40:25.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.189+0000 7f0d7dc32640 1 -- 172.21.15.111:0/3899072603 wait complete. 2023-12-04T05:40:25.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:25.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 Processor -- start 2023-12-04T05:40:25.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 -- start start 2023-12-04T05:40:25.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d860 0x7f532c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f532c0d0010 0x7f532c160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c15ddc0 con 0x7f532c15d480 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0d04a0 con 0x7f532c0d0010 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.303+0000 7f5331bd5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c15e0a0 con 0x7f532c15d860 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532bfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f532c0d0010 0x7f532c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f532c15d480 0x7f532c0d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58216/0 (socket says 172.21.15.111:58216) 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532b7fe640 1 -- 172.21.15.111:0/2826913140 learned_addr learned my addr 172.21.15.111:0/2826913140 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532bfff640 1 -- 172.21.15.111:0/2826913140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d860 msgr2=0x7f532c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532affd640 1 --2- 172.21.15.111:0/2826913140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d860 0x7f532c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532a7fc640 1 -- 172.21.15.111:0/2826913140 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567852177 0 0) 0x7f532c15ddc0 con 0x7f532c15d480 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532bfff640 1 --2- 172.21.15.111:0/2826913140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d860 0x7f532c0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532bfff640 1 -- 172.21.15.111:0/2826913140 >> v1:172.21.15.111:6789/0 conn(0x7f532c15d480 legacy=0x7f532c0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.304+0000 7f532bfff640 1 -- 172.21.15.111:0/2826913140 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c15e380 con 0x7f532c0d0010 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f532bfff640 1 --2- 172.21.15.111:0/2826913140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c160270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5324009d90 tx=0x7f532402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa243c5fb98b5dd server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f532a7fc640 1 -- 172.21.15.111:0/2826913140 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532402cc50 con 0x7f532c0d0010 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f532a7fc640 1 -- 172.21.15.111:0/2826913140 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532402cdb0 con 0x7f532c0d0010 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f532a7fc640 1 -- 172.21.15.111:0/2826913140 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324034660 con 0x7f532c0d0010 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 -- 172.21.15.111:0/2826913140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 msgr2=0x7f532c160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 --2- 172.21.15.111:0/2826913140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c160270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5324009d90 tx=0x7f532402bea0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 -- 172.21.15.111:0/2826913140 shutdown_connections 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 --2- 172.21.15.111:0/2826913140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c160270 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 --2- 172.21.15.111:0/2826913140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d860 0x7f532c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 -- 172.21.15.111:0/2826913140 >> 172.21.15.111:0/2826913140 conn(0x7f532c09c540 msgr2=0x7f532c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 -- 172.21.15.111:0/2826913140 shutdown_connections 2023-12-04T05:40:25.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.305+0000 7f5331bd5640 1 -- 172.21.15.111:0/2826913140 wait complete. 2023-12-04T05:40:25.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.306+0000 7f5331bd5640 1 Processor -- start 2023-12-04T05:40:25.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 -- start start 2023-12-04T05:40:25.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c14c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d480 0x7f532c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532c15d860 0x7f532c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c150b80 con 0x7f532c0d0010 2023-12-04T05:40:25.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0a0f90 con 0x7f532c15d860 2023-12-04T05:40:25.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f5331bd5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c14ff70 con 0x7f532c15d480 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f532b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c14c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f532b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c14c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60578/0 (socket says 172.21.15.111:60578) 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.307+0000 7f532b7fe640 1 -- 172.21.15.111:0/3603124996 learned_addr learned my addr 172.21.15.111:0/3603124996 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532bfff640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532c15d860 0x7f532c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532b7fe640 1 -- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d480 msgr2=0x7f532c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532b7fe640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d480 0x7f532c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532b7fe640 1 -- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532c15d860 msgr2=0x7f532c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532b7fe640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532c15d860 0x7f532c151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532b7fe640 1 -- 172.21.15.111:0/3603124996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324009a70 con 0x7f532c0d0010 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f532b7fe640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c14c070 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c4d0 tx=0x7f531c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0074e0 con 0x7f532c0d0010 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c007640 con 0x7f532c0d0010 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c150250 con 0x7f532c0d0010 2023-12-04T05:40:25.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c004a20 con 0x7f532c0d0010 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.308+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c150810 con 0x7f532c0d0010 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.309+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c067980 con 0x7f532c0d0010 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.309+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f531c004b80 con 0x7f532c0d0010 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.310+0000 7f5313fff640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f531806b2d0 0x7f531806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.310+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f531c089060 con 0x7f532c0d0010 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.310+0000 7f532affd640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f531806b2d0 0x7f531806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.310+0000 7f532affd640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f531806b2d0 0x7f531806d790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f532c14f190 tx=0x7f5320008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.314+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f531c053f60 con 0x7f532c0d0010 2023-12-04T05:40:25.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.481+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f532c069360 con 0x7f532c0d0010 2023-12-04T05:40:25.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.482+0000 7f5313fff640 1 -- 172.21.15.111:0/3603124996 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f531c018020 con 0x7f532c0d0010 2023-12-04T05:40:25.484 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:25.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f531806b2d0 msgr2=0x7f531806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f531806b2d0 0x7f531806d790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f532c14f190 tx=0x7f5320008040 comp rx=0 tx=0).stop 2023-12-04T05:40:25.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 msgr2=0x7f532c14c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c14c070 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c4d0 tx=0x7f531c00c990 comp rx=0 tx=0).stop 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 shutdown_connections 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f531806b2d0 0x7f531806d790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532c15d860 0x7f532c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f532c15d480 0x7f532c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 --2- 172.21.15.111:0/3603124996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f532c0d0010 0x7f532c14c070 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 >> 172.21.15.111:0/3603124996 conn(0x7f532c09c540 msgr2=0x7f532c0b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 shutdown_connections 2023-12-04T05:40:25.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.484+0000 7f5331bd5640 1 -- 172.21.15.111:0/3603124996 wait complete. 2023-12-04T05:40:25.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:25.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.598+0000 7f89f786b640 1 Processor -- start 2023-12-04T05:40:25.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.598+0000 7f89f786b640 1 -- start start 2023-12-04T05:40:25.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.598+0000 7f89f786b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f89f013f640 0x7f89f013fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.598+0000 7f89f786b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89f0071d20 0x7f89f0072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f786b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f013dd10 con 0x7f89f013cee0 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f786b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f0072670 con 0x7f89f0071d20 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f786b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f013d2c0 con 0x7f89f013f640 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89f0071d20 0x7f89f0072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89f0071d20 0x7f89f0072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60590/0 (socket says 172.21.15.111:60590) 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 -- 172.21.15.111:0/1360397219 learned_addr learned my addr 172.21.15.111:0/1360397219 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 -- 172.21.15.111:0/1360397219 >> v2:172.21.15.192:3300/0 conn(0x7f89f013f640 msgr2=0x7f89f013fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 --2- 172.21.15.111:0/1360397219 >> v2:172.21.15.192:3300/0 conn(0x7f89f013f640 0x7f89f013fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 -- 172.21.15.111:0/1360397219 >> v1:172.21.15.192:6789/0 conn(0x7f89f013cee0 legacy=0x7f89f013d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:25.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.599+0000 7f89f5e08640 1 -- 172.21.15.111:0/1360397219 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f01400e0 con 0x7f89f0071d20 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f5e08640 1 --2- 172.21.15.111:0/1360397219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f0071d20 0x7f89f0072130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f89f006bdd0 tx=0x7f89ec02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb700340200e11aa server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89e7fff640 1 -- 172.21.15.111:0/1360397219 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec031750 con 0x7f89f0071d20 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89e7fff640 1 -- 172.21.15.111:0/1360397219 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec0318b0 con 0x7f89f0071d20 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89e7fff640 1 -- 172.21.15.111:0/1360397219 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec031c40 con 0x7f89f0071d20 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 -- 172.21.15.111:0/1360397219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f0071d20 msgr2=0x7f89f0072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 --2- 172.21.15.111:0/1360397219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f0071d20 0x7f89f0072130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f89f006bdd0 tx=0x7f89ec02b7f0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 -- 172.21.15.111:0/1360397219 shutdown_connections 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 --2- 172.21.15.111:0/1360397219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f0071d20 0x7f89f0072130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 --2- 172.21.15.111:0/1360397219 >> v2:172.21.15.192:3300/0 conn(0x7f89f013f640 0x7f89f013fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 -- 172.21.15.111:0/1360397219 >> 172.21.15.111:0/1360397219 conn(0x7f89f00b0cf0 msgr2=0x7f89f00b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 -- 172.21.15.111:0/1360397219 shutdown_connections 2023-12-04T05:40:25.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.600+0000 7f89f786b640 1 -- 172.21.15.111:0/1360397219 wait complete. 2023-12-04T05:40:25.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.601+0000 7f89f786b640 1 Processor -- start 2023-12-04T05:40:25.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 -- start start 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0071d20 0x7f89f0136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f013cee0 0x7f89f013b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 0x7f89f013ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0167780 con 0x7f89f013f640 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f013c640 con 0x7f89f013cee0 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f786b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f01668d0 con 0x7f89f0071d20 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f5e08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 0x7f89f013ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f5e08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 0x7f89f013ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60600/0 (socket says 172.21.15.111:60600) 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.602+0000 7f89f5e08640 1 -- 172.21.15.111:0/901232231 learned_addr learned my addr 172.21.15.111:0/901232231 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f5e08640 1 -- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0071d20 msgr2=0x7f89f0136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f5e08640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0071d20 0x7f89f0136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f5e08640 1 -- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f013cee0 msgr2=0x7f89f013b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f4e06640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f013cee0 0x7f89f013b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f5e08640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f013cee0 0x7f89f013b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f5e08640 1 -- 172.21.15.111:0/901232231 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0094d0 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f4e06640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f013cee0 0x7f89f013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f5e08640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 0x7f89f013ba60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f89ec02b830 tx=0x7f89ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec033c80 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec033e00 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f0166b70 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec03d6b0 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.603+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f0167130 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.604+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f01391a0 con 0x7f89f013f640 2023-12-04T05:40:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.605+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f89ec03a040 con 0x7f89f013f640 2023-12-04T05:40:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.605+0000 7f89e5ffb640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f89d406b2a0 0x7f89d406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.605+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89ec0abc80 con 0x7f89f013f640 2023-12-04T05:40:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.605+0000 7f89f5607640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f89d406b2a0 0x7f89d406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.605+0000 7f89f5607640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f89d406b2a0 0x7f89d406d760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89e8001c60 tx=0x7f89e8007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.609+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89ec076af0 con 0x7f89f013f640 2023-12-04T05:40:25.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.777+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89f01522f0 con 0x7f89f013f640 2023-12-04T05:40:25.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.778+0000 7f89e5ffb640 1 -- 172.21.15.111:0/901232231 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89ec07a110 con 0x7f89f013f640 2023-12-04T05:40:25.779 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:25.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f89d406b2a0 msgr2=0x7f89d406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f89d406b2a0 0x7f89d406d760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89e8001c60 tx=0x7f89e8007d10 comp rx=0 tx=0).stop 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 msgr2=0x7f89f013ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 0x7f89f013ba60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f89ec02b830 tx=0x7f89ec0046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 shutdown_connections 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f89d406b2a0 0x7f89d406d760 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89f013f640 0x7f89f013ba60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f013cee0 0x7f89f013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 --2- 172.21.15.111:0/901232231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0071d20 0x7f89f0136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 >> 172.21.15.111:0/901232231 conn(0x7f89f00b0cf0 msgr2=0x7f89f00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 shutdown_connections 2023-12-04T05:40:25.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.780+0000 7f89f786b640 1 -- 172.21.15.111:0/901232231 wait complete. 2023-12-04T05:40:25.793 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:40:25.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.892+0000 7f3804b9e640 1 Processor -- start 2023-12-04T05:40:25.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.892+0000 7f3804b9e640 1 -- start start 2023-12-04T05:40:25.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f3804b9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3800160600 0x7f38000d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f3804b9e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38000cf840 0x7f38000cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f3804b9e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38000d2020 con 0x7f38001609e0 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f3804b9e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38000d0ad0 con 0x7f38000cf840 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f3804b9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000d01a0 con 0x7f3800160600 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37fffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38000cf840 0x7f38000cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37feffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f38001609e0 0x7f38000ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58232/0 (socket says 172.21.15.111:58232) 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37feffd640 1 -- 172.21.15.111:0/1738769675 learned_addr learned my addr 172.21.15.111:0/1738769675 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37fffff640 1 -- 172.21.15.111:0/1738769675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3800160600 msgr2=0x7f38000d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37fffff640 1 --2- 172.21.15.111:0/1738769675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3800160600 0x7f38000d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37fffff640 1 -- 172.21.15.111:0/1738769675 >> v1:172.21.15.111:6789/0 conn(0x7f38001609e0 legacy=0x7f38000ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37fe7fc640 1 -- 172.21.15.111:0/1738769675 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797799748 0 0) 0x7f37f4001110 con 0x7f38001609e0 2023-12-04T05:40:25.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.893+0000 7f37fffff640 1 -- 172.21.15.111:0/1738769675 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38000ced30 con 0x7f38000cf840 2023-12-04T05:40:25.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.894+0000 7f37fffff640 1 --2- 172.21.15.111:0/1738769675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38000cf840 0x7f38000cfc60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009d90 tx=0x7f37f802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35d39d90d908f7e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.894+0000 7f37fe7fc640 1 -- 172.21.15.111:0/1738769675 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f802cbc0 con 0x7f38000cf840 2023-12-04T05:40:25.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.894+0000 7f37fe7fc640 1 -- 172.21.15.111:0/1738769675 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f802cd20 con 0x7f38000cf840 2023-12-04T05:40:25.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.894+0000 7f37fe7fc640 1 -- 172.21.15.111:0/1738769675 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8034660 con 0x7f38000cf840 2023-12-04T05:40:25.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 -- 172.21.15.111:0/1738769675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38000cf840 msgr2=0x7f38000cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 --2- 172.21.15.111:0/1738769675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38000cf840 0x7f38000cfc60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009d90 tx=0x7f37f802be10 comp rx=0 tx=0).stop 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 -- 172.21.15.111:0/1738769675 shutdown_connections 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 --2- 172.21.15.111:0/1738769675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38000cf840 0x7f38000cfc60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 --2- 172.21.15.111:0/1738769675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3800160600 0x7f38000d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 -- 172.21.15.111:0/1738769675 >> 172.21.15.111:0/1738769675 conn(0x7f38000d2f00 msgr2=0x7f38000d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 -- 172.21.15.111:0/1738769675 shutdown_connections 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.895+0000 7f3804b9e640 1 -- 172.21.15.111:0/1738769675 wait complete. 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.896+0000 7f3804b9e640 1 Processor -- start 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.896+0000 7f3804b9e640 1 -- start start 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f3804b9e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38000cf840 0x7f3800158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f3804b9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 0x7f3800158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f3804b9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38001609e0 0x7f38000c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f3804b9e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800158760 con 0x7f3800160600 2023-12-04T05:40:25.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f3804b9e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38001f30d0 con 0x7f38000cf840 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f3804b9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3800159370 con 0x7f38001609e0 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 0x7f3800158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 0x7f3800158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60622/0 (socket says 172.21.15.111:60622) 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 -- 172.21.15.111:0/105660272 learned_addr learned my addr 172.21.15.111:0/105660272 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37ff7fe640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38000cf840 0x7f3800158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 -- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38001609e0 msgr2=0x7f38000c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38001609e0 0x7f38000c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 -- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38000cf840 msgr2=0x7f3800158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38000cf840 0x7f3800158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.897+0000 7f37feffd640 1 -- 172.21.15.111:0/105660272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009a70 con 0x7f3800160600 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.898+0000 7f37feffd640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 0x7f3800158e30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f37f400c7b0 tx=0x7f37f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.898+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40072d0 con 0x7f3800160600 2023-12-04T05:40:25.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.898+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f4007450 con 0x7f3800160600 2023-12-04T05:40:25.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.898+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40047e0 con 0x7f3800160600 2023-12-04T05:40:25.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.898+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38000c7930 con 0x7f3800160600 2023-12-04T05:40:25.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.898+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38000c7ef0 con 0x7f3800160600 2023-12-04T05:40:25.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.899+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3800069360 con 0x7f3800160600 2023-12-04T05:40:25.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.899+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f37f4004960 con 0x7f3800160600 2023-12-04T05:40:25.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.900+0000 7f37e7fff640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f37ec06b2a0 0x7f37ec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:25.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.900+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37f4088bb0 con 0x7f3800160600 2023-12-04T05:40:25.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.900+0000 7f37ff7fe640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f37ec06b2a0 0x7f37ec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:25.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.900+0000 7f37ff7fe640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f37ec06b2a0 0x7f37ec06d760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37f000a810 tx=0x7f37f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:25.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:25.904+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37f4053ab0 con 0x7f3800160600 2023-12-04T05:40:26.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.081+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38000d2020 con 0x7f3800160600 2023-12-04T05:40:26.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.081+0000 7f37e7fff640 1 -- 172.21.15.111:0/105660272 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f38000d2020 con 0x7f3800160600 2023-12-04T05:40:26.083 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:40:26.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f37ec06b2a0 msgr2=0x7f37ec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f37ec06b2a0 0x7f37ec06d760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37f000a810 tx=0x7f37f0008040 comp rx=0 tx=0).stop 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 msgr2=0x7f3800158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 0x7f3800158e30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f37f400c7b0 tx=0x7f37f400cc70 comp rx=0 tx=0).stop 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 shutdown_connections 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f37ec06b2a0 0x7f37ec06d760 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38001609e0 0x7f38000c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800160600 0x7f3800158e30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 --2- 172.21.15.111:0/105660272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38000cf840 0x7f3800158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 >> 172.21.15.111:0/105660272 conn(0x7f38000d2f00 msgr2=0x7f38000b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 shutdown_connections 2023-12-04T05:40:26.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.083+0000 7f3804b9e640 1 -- 172.21.15.111:0/105660272 wait complete. 2023-12-04T05:40:26.095 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:26.095 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:26.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:26.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.248+0000 7fba5bf41640 1 Processor -- start 2023-12-04T05:40:26.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5bf41640 1 -- start start 2023-12-04T05:40:26.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5bf41640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba540d1780 0x7fba540d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5bf41640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fba540b20a0 0x7fba540d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5bf41640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540cdcb0 con 0x7fba540b2ad0 2023-12-04T05:40:26.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5bf41640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba540d20a0 con 0x7fba540d1780 2023-12-04T05:40:26.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5bf41640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba540d39e0 con 0x7fba540b20a0 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba59cdd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba540d1780 0x7fba540d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5a4de640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fba540b2ad0 0x7fba540ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58242/0 (socket says 172.21.15.111:58242) 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.249+0000 7fba5a4de640 1 -- 172.21.15.111:0/735193072 learned_addr learned my addr 172.21.15.111:0/735193072 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.250+0000 7fba59cdd640 1 -- 172.21.15.111:0/735193072 >> v2:172.21.15.192:3300/0 conn(0x7fba540b20a0 msgr2=0x7fba540d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.250+0000 7fba59cdd640 1 --2- 172.21.15.111:0/735193072 >> v2:172.21.15.192:3300/0 conn(0x7fba540b20a0 0x7fba540d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.250+0000 7fba59cdd640 1 -- 172.21.15.111:0/735193072 >> v1:172.21.15.111:6789/0 conn(0x7fba540b2ad0 legacy=0x7fba540ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.250+0000 7fba59cdd640 1 -- 172.21.15.111:0/735193072 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540d3cc0 con 0x7fba540d1780 2023-12-04T05:40:26.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.250+0000 7fba58cdb640 1 -- 172.21.15.111:0/735193072 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104923195 0 0) 0x7fba540cdcb0 con 0x7fba540b2ad0 2023-12-04T05:40:26.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.250+0000 7fba59cdd640 1 --2- 172.21.15.111:0/735193072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540d1780 0x7fba540d01e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fba540d0c50 tx=0x7fba5002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c0e6c52bda3c54 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba58cdb640 1 -- 172.21.15.111:0/735193072 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5002cbf0 con 0x7fba540d1780 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba58cdb640 1 -- 172.21.15.111:0/735193072 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5002cd50 con 0x7fba540d1780 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba58cdb640 1 -- 172.21.15.111:0/735193072 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50034660 con 0x7fba540d1780 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 -- 172.21.15.111:0/735193072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540d1780 msgr2=0x7fba540d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 --2- 172.21.15.111:0/735193072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540d1780 0x7fba540d01e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fba540d0c50 tx=0x7fba5002be40 comp rx=0 tx=0).stop 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 -- 172.21.15.111:0/735193072 shutdown_connections 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 --2- 172.21.15.111:0/735193072 >> v2:172.21.15.192:3300/0 conn(0x7fba540b20a0 0x7fba540d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 --2- 172.21.15.111:0/735193072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540d1780 0x7fba540d01e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 -- 172.21.15.111:0/735193072 >> 172.21.15.111:0/735193072 conn(0x7fba5409c4e0 msgr2=0x7fba540d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 -- 172.21.15.111:0/735193072 shutdown_connections 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.251+0000 7fba5bf41640 1 -- 172.21.15.111:0/735193072 wait complete. 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.252+0000 7fba5bf41640 1 Processor -- start 2023-12-04T05:40:26.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.252+0000 7fba5bf41640 1 -- start start 2023-12-04T05:40:26.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba5bf41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 0x7fba5415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba5bf41640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba540b2ad0 0x7fba540b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba5bf41640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba540d1780 0x7fba540b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba5bf41640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b76c0 con 0x7fba540b20a0 2023-12-04T05:40:26.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba5bf41640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b67d0 con 0x7fba540d1780 2023-12-04T05:40:26.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba5bf41640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b6ab0 con 0x7fba540b2ad0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 0x7fba5415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 0x7fba5415bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60638/0 (socket says 172.21.15.111:60638) 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 -- 172.21.15.111:0/3816723322 learned_addr learned my addr 172.21.15.111:0/3816723322 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 -- 172.21.15.111:0/3816723322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba540b2ad0 msgr2=0x7fba540b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba540b2ad0 0x7fba540b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 -- 172.21.15.111:0/3816723322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba540d1780 msgr2=0x7fba540b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba540d1780 0x7fba540b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 -- 172.21.15.111:0/3816723322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba50009a70 con 0x7fba540b20a0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.253+0000 7fba59cdd640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 0x7fba5415bde0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fba50037040 tx=0x7fba50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.254+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5003e610 con 0x7fba540b20a0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.254+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5003e770 con 0x7fba540b20a0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.254+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba540b6d90 con 0x7fba540b20a0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.254+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5003eb20 con 0x7fba540b20a0 2023-12-04T05:40:26.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.254+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba540b7350 con 0x7fba540b20a0 2023-12-04T05:40:26.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.255+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540693d0 con 0x7fba540b20a0 2023-12-04T05:40:26.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.255+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fba5003b050 con 0x7fba540b20a0 2023-12-04T05:40:26.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.255+0000 7fba4a7fc640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba3806b2a0 0x7fba3806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.256+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba500ac6a0 con 0x7fba540b20a0 2023-12-04T05:40:26.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.256+0000 7fba594dc640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba3806b2a0 0x7fba3806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.256+0000 7fba594dc640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba3806b2a0 0x7fba3806d760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba541f6370 tx=0x7fba44006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.260+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba50077510 con 0x7fba540b20a0 2023-12-04T05:40:26.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.428+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba540d40b0 con 0x7fba540b20a0 2023-12-04T05:40:26.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.430+0000 7fba4a7fc640 1 -- 172.21.15.111:0/3816723322 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fba5007ab30 con 0x7fba540b20a0 2023-12-04T05:40:26.431 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:26.431 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:26.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.431+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba3806b2a0 msgr2=0x7fba3806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.431+0000 7fba5bf41640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba3806b2a0 0x7fba3806d760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba541f6370 tx=0x7fba44006bf0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.431+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 msgr2=0x7fba5415bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.431+0000 7fba5bf41640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 0x7fba5415bde0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fba50037040 tx=0x7fba50004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:26.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 shutdown_connections 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba3806b2a0 0x7fba3806d760 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba540d1780 0x7fba540b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba540b2ad0 0x7fba540b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 --2- 172.21.15.111:0/3816723322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba540b20a0 0x7fba5415bde0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 >> 172.21.15.111:0/3816723322 conn(0x7fba5409c4e0 msgr2=0x7fba5406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 shutdown_connections 2023-12-04T05:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.432+0000 7fba5bf41640 1 -- 172.21.15.111:0/3816723322 wait complete. 2023-12-04T05:40:26.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:26.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:26.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.545+0000 7fb400790640 1 Processor -- start 2023-12-04T05:40:26.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.545+0000 7fb400790640 1 -- start start 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb400790640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3f815b0c0 0x7fb3f815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb400790640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f8072250 0x7fb3f80cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb400790640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f80cd320 con 0x7fb3f80ce940 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb400790640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f80ced80 con 0x7fb3f815b0c0 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb400790640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f80cbee0 con 0x7fb3f8072250 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fe52c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3f815b0c0 0x7fb3f815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fdd2b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb3f80ce940 0x7fb3f8071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58258/0 (socket says 172.21.15.111:58258) 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fe52c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3f815b0c0 0x7fb3f815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60650/0 (socket says 172.21.15.111:60650) 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fdd2b640 1 -- 172.21.15.111:0/3551992969 learned_addr learned my addr 172.21.15.111:0/3551992969 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fe52c640 1 -- 172.21.15.111:0/3551992969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f8072250 msgr2=0x7fb3f80cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:26.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fed2d640 1 --2- 172.21.15.111:0/3551992969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f8072250 0x7fb3f80cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fe52c640 1 --2- 172.21.15.111:0/3551992969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f8072250 0x7fb3f80cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fe52c640 1 -- 172.21.15.111:0/3551992969 >> v1:172.21.15.111:6789/0 conn(0x7fb3f80ce940 legacy=0x7fb3f8071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fd52a640 1 -- 172.21.15.111:0/3551992969 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017193777 0 0) 0x7fb3f80cd320 con 0x7fb3f80ce940 2023-12-04T05:40:26.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.546+0000 7fb3fe52c640 1 -- 172.21.15.111:0/3551992969 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f80cc710 con 0x7fb3f815b0c0 2023-12-04T05:40:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.547+0000 7fb3fe52c640 1 --2- 172.21.15.111:0/3551992969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f815b0c0 0x7fb3f815ed10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40097a0 tx=0x7fb3f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130d263f3c5fe20d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.547+0000 7fb3fd52a640 1 -- 172.21.15.111:0/3551992969 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f402e070 con 0x7fb3f815b0c0 2023-12-04T05:40:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.547+0000 7fb3fd52a640 1 -- 172.21.15.111:0/3551992969 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f40315a0 con 0x7fb3f815b0c0 2023-12-04T05:40:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.547+0000 7fb3fd52a640 1 -- 172.21.15.111:0/3551992969 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031930 con 0x7fb3f815b0c0 2023-12-04T05:40:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 -- 172.21.15.111:0/3551992969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f815b0c0 msgr2=0x7fb3f815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 --2- 172.21.15.111:0/3551992969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f815b0c0 0x7fb3f815ed10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40097a0 tx=0x7fb3f402b840 comp rx=0 tx=0).stop 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 -- 172.21.15.111:0/3551992969 shutdown_connections 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 --2- 172.21.15.111:0/3551992969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f8072250 0x7fb3f80cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 --2- 172.21.15.111:0/3551992969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f815b0c0 0x7fb3f815ed10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 -- 172.21.15.111:0/3551992969 >> 172.21.15.111:0/3551992969 conn(0x7fb3f80b0cf0 msgr2=0x7fb3f80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 -- 172.21.15.111:0/3551992969 shutdown_connections 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.548+0000 7fb400790640 1 -- 172.21.15.111:0/3551992969 wait complete. 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.549+0000 7fb400790640 1 Processor -- start 2023-12-04T05:40:26.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.549+0000 7fb400790640 1 -- start start 2023-12-04T05:40:26.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb400790640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3f8072250 0x7fb3f8147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb400790640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 0x7fb3f8158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb400790640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f815b0c0 0x7fb3f8158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb400790640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81591a0 con 0x7fb3f80ce940 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb400790640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8157b10 con 0x7fb3f8072250 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb400790640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8157df0 con 0x7fb3f815b0c0 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 0x7fb3f8158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 0x7fb3f8158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60652/0 (socket says 172.21.15.111:60652) 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 -- 172.21.15.111:0/996602340 learned_addr learned my addr 172.21.15.111:0/996602340 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fe52c640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3f8072250 0x7fb3f8147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fed2d640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f815b0c0 0x7fb3f8158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 -- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f815b0c0 msgr2=0x7fb3f8158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f815b0c0 0x7fb3f8158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 -- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3f8072250 msgr2=0x7fb3f8147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3f8072250 0x7fb3f8147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 -- 172.21.15.111:0/996602340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009400 con 0x7fb3f80ce940 2023-12-04T05:40:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.550+0000 7fb3fdd2b640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 0x7fb3f8158570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800c4d0 tx=0x7fb3e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.552+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80074e0 con 0x7fb3f80ce940 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.552+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8007660 con 0x7fb3f80ce940 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.552+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8004a20 con 0x7fb3f80ce940 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.552+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f80aca10 con 0x7fb3f80ce940 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.552+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81f4110 con 0x7fb3f80ce940 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.553+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f80ac870 con 0x7fb3f80ce940 2023-12-04T05:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.554+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb3e8004b80 con 0x7fb3f80ce940 2023-12-04T05:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.554+0000 7fb3eeffd640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3dc06b2a0 0x7fb3dc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.554+0000 7fb3fe52c640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3dc06b2a0 0x7fb3dc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.554+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3e8088d10 con 0x7fb3f80ce940 2023-12-04T05:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.555+0000 7fb3fe52c640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3dc06b2a0 0x7fb3dc06d760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4036000 tx=0x7fb3f403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.558+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3f80ac870 con 0x7fb3f80ce940 2023-12-04T05:40:26.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.725+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3f80ac870 con 0x7fb3f80ce940 2023-12-04T05:40:26.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.725+0000 7fb3eeffd640 1 -- 172.21.15.111:0/996602340 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3f80ac870 con 0x7fb3f80ce940 2023-12-04T05:40:26.727 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:26.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3dc06b2a0 msgr2=0x7fb3dc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3dc06b2a0 0x7fb3dc06d760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4036000 tx=0x7fb3f403d040 comp rx=0 tx=0).stop 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 msgr2=0x7fb3f8158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 0x7fb3f8158570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800c4d0 tx=0x7fb3e800c990 comp rx=0 tx=0).stop 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 shutdown_connections 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3dc06b2a0 0x7fb3dc06d760 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3f815b0c0 0x7fb3f8158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3f80ce940 0x7fb3f8158570 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 --2- 172.21.15.111:0/996602340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3f8072250 0x7fb3f8147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 >> 172.21.15.111:0/996602340 conn(0x7fb3f80b0cf0 msgr2=0x7fb3f806c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 shutdown_connections 2023-12-04T05:40:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.727+0000 7fb400790640 1 -- 172.21.15.111:0/996602340 wait complete. 2023-12-04T05:40:26.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:26.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.840+0000 7f4ad2e83640 1 Processor -- start 2023-12-04T05:40:26.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.840+0000 7f4ad2e83640 1 -- start start 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad2e83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4acc070c10 0x7f4acc0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad2e83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad2e83640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc0cd790 con 0x7f4acc0cd020 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad2e83640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4acc159c20 con 0x7f4acc070c10 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad2e83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0ced70 con 0x7f4acc070ff0 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4acc070c10 0x7f4acc0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4acc070c10 0x7f4acc0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60658/0 (socket says 172.21.15.111:60658) 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/617270446 learned_addr learned my addr 172.21.15.111:0/617270446 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/617270446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 msgr2=0x7f4acc0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:26.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 --2- 172.21.15.111:0/617270446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/617270446 >> v1:172.21.15.192:6789/0 conn(0x7f4acc0cd020 legacy=0x7f4acc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:26.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.841+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/617270446 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc0cda70 con 0x7f4acc070c10 2023-12-04T05:40:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.842+0000 7f4ad0c1f640 1 --2- 172.21.15.111:0/617270446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc0ce830 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000aa10 tx=0x7f4ac002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c424e69f94b36de server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.842+0000 7f4acb7fe640 1 -- 172.21.15.111:0/617270446 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac002db80 con 0x7f4acc070c10 2023-12-04T05:40:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.842+0000 7f4acb7fe640 1 -- 172.21.15.111:0/617270446 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac002dce0 con 0x7f4acc070c10 2023-12-04T05:40:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.842+0000 7f4acb7fe640 1 -- 172.21.15.111:0/617270446 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0035660 con 0x7f4acc070c10 2023-12-04T05:40:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 -- 172.21.15.111:0/617270446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 msgr2=0x7f4acc0ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/617270446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc0ce830 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000aa10 tx=0x7f4ac002cdd0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 -- 172.21.15.111:0/617270446 shutdown_connections 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/617270446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc0714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/617270446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc0ce830 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 -- 172.21.15.111:0/617270446 >> 172.21.15.111:0/617270446 conn(0x7f4acc0b0cf0 msgr2=0x7f4acc0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 -- 172.21.15.111:0/617270446 shutdown_connections 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.843+0000 7f4ad2e83640 1 -- 172.21.15.111:0/617270446 wait complete. 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.844+0000 7f4ad2e83640 1 Processor -- start 2023-12-04T05:40:26.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.844+0000 7f4ad2e83640 1 -- start start 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4acc0cd020 0x7f4acc0aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0aaed0 con 0x7f4acc070c10 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc1f49b0 con 0x7f4acc0cd020 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc1f4c90 con 0x7f4acc070ff0 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc1f5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60674/0 (socket says 172.21.15.111:60674) 2023-12-04T05:40:26.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/3693867759 learned_addr learned my addr 172.21.15.111:0/3693867759 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 msgr2=0x7f4acc13e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad1420640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4acc0cd020 0x7f4acc0aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4acbfff640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc13e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4acc0cd020 msgr2=0x7f4acc0aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4acc0cd020 0x7f4acc0aa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 -- 172.21.15.111:0/3693867759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac000a670 con 0x7f4acc070c10 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad1420640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4acc0cd020 0x7f4acc0aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad0c1f640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc1f5490 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000a1c0 tx=0x7f4ac0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0040650 con 0x7f4acc070c10 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac00407d0 con 0x7f4acc070c10 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc1f4f70 con 0x7f4acc070c10 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0040b80 con 0x7f4acc070c10 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.845+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc0aab70 con 0x7f4acc070c10 2023-12-04T05:40:26.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.846+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc070a80 con 0x7f4acc070c10 2023-12-04T05:40:26.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.847+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4ac003c070 con 0x7f4acc070c10 2023-12-04T05:40:26.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.847+0000 7f4ac97fa640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ab006b2a0 0x7f4ab006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:26.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.847+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ac00ac480 con 0x7f4acc070c10 2023-12-04T05:40:26.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.847+0000 7f4acbfff640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ab006b2a0 0x7f4ab006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:26.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.848+0000 7f4acbfff640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ab006b2a0 0x7f4ab006d760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4acc1593f0 tx=0x7f4ab8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:26.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:26.851+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ac00772f0 con 0x7f4acc070c10 2023-12-04T05:40:27.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.017+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4acc069360 con 0x7f4acc070c10 2023-12-04T05:40:27.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.018+0000 7f4ac97fa640 1 -- 172.21.15.111:0/3693867759 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ac007a910 con 0x7f4acc070c10 2023-12-04T05:40:27.019 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:27.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.019+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ab006b2a0 msgr2=0x7f4ab006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.019+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ab006b2a0 0x7f4ab006d760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4acc1593f0 tx=0x7f4ab8008040 comp rx=0 tx=0).stop 2023-12-04T05:40:27.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.019+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 msgr2=0x7f4acc1f5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.019+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc1f5490 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000a1c0 tx=0x7f4ac0004730 comp rx=0 tx=0).stop 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 shutdown_connections 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ab006b2a0 0x7f4ab006d760 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4acc0cd020 0x7f4acc0aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4acc070ff0 0x7f4acc13e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 --2- 172.21.15.111:0/3693867759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4acc070c10 0x7f4acc1f5490 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 >> 172.21.15.111:0/3693867759 conn(0x7f4acc0b0cf0 msgr2=0x7f4acc13f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 shutdown_connections 2023-12-04T05:40:27.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.020+0000 7f4ad2e83640 1 -- 172.21.15.111:0/3693867759 wait complete. 2023-12-04T05:40:27.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:27.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 Processor -- start 2023-12-04T05:40:27.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 -- start start 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813cee0 0x7fe62813d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe628072430 0x7fe62813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62813d2c0 con 0x7fe62813f640 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6281400e0 con 0x7fe628072430 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.134+0000 7fe62dde5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281403c0 con 0x7fe62813cee0 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe627fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe628072430 0x7fe62813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe626ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe62813f640 0x7fe628071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58274/0 (socket says 172.21.15.111:58274) 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe627fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe628072430 0x7fe62813fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60686/0 (socket says 172.21.15.111:60686) 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe626ffd640 1 -- 172.21.15.111:0/2846247009 learned_addr learned my addr 172.21.15.111:0/2846247009 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe6277fe640 1 --2- 172.21.15.111:0/2846247009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813cee0 0x7fe62813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe627fff640 1 -- 172.21.15.111:0/2846247009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813cee0 msgr2=0x7fe62813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe627fff640 1 --2- 172.21.15.111:0/2846247009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813cee0 0x7fe62813d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe627fff640 1 -- 172.21.15.111:0/2846247009 >> v1:172.21.15.111:6789/0 conn(0x7fe62813f640 legacy=0x7fe628071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe627fff640 1 -- 172.21.15.111:0/2846247009 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280cdb50 con 0x7fe628072430 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.135+0000 7fe6267fc640 1 -- 172.21.15.111:0/2846247009 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080925172 0 0) 0x7fe62813d2c0 con 0x7fe62813f640 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe627fff640 1 --2- 172.21.15.111:0/2846247009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe628072430 0x7fe62813fa80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe62002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57fc9abbb26cb9e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe6267fc640 1 -- 172.21.15.111:0/2846247009 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62002cb30 con 0x7fe628072430 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe6267fc640 1 -- 172.21.15.111:0/2846247009 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62002cc90 con 0x7fe628072430 2023-12-04T05:40:27.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe6267fc640 1 -- 172.21.15.111:0/2846247009 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620034440 con 0x7fe628072430 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 -- 172.21.15.111:0/2846247009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe628072430 msgr2=0x7fe62813fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2846247009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe628072430 0x7fe62813fa80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe620009d90 tx=0x7fe62002bd80 comp rx=0 tx=0).stop 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 -- 172.21.15.111:0/2846247009 shutdown_connections 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2846247009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe628072430 0x7fe62813fa80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2846247009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813cee0 0x7fe62813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 -- 172.21.15.111:0/2846247009 >> 172.21.15.111:0/2846247009 conn(0x7fe6280b0cf0 msgr2=0x7fe6280b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:27.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 -- 172.21.15.111:0/2846247009 shutdown_connections 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.136+0000 7fe62dde5640 1 -- 172.21.15.111:0/2846247009 wait complete. 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.137+0000 7fe62dde5640 1 Processor -- start 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 -- start start 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe628072430 0x7fe6280b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 0x7fe6281f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813f640 0x7fe62813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280b3620 con 0x7fe62813cee0 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62813c170 con 0x7fe628072430 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe62dde5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280ac910 con 0x7fe62813f640 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.138+0000 7fe626ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 0x7fe6281f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 0x7fe6281f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60694/0 (socket says 172.21.15.111:60694) 2023-12-04T05:40:27.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 -- 172.21.15.111:0/2117571321 learned_addr learned my addr 172.21.15.111:0/2117571321 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe6277fe640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe628072430 0x7fe6280b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe627fff640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813f640 0x7fe62813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 -- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813f640 msgr2=0x7fe62813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813f640 0x7fe62813b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 -- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe628072430 msgr2=0x7fe6280b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe628072430 0x7fe6280b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 -- 172.21.15.111:0/2117571321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620009a70 con 0x7fe62813cee0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe626ffd640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 0x7fe6281f2520 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00c430 tx=0x7fe61c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0074e0 con 0x7fe62813cee0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c007660 con 0x7fe62813cee0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62813c3c0 con 0x7fe62813cee0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004a20 con 0x7fe62813cee0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.139+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6280ace90 con 0x7fe62813cee0 2023-12-04T05:40:27.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.140+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe628069360 con 0x7fe62813cee0 2023-12-04T05:40:27.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.140+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe61c004b80 con 0x7fe62813cee0 2023-12-04T05:40:27.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.141+0000 7fe60ffff640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe61406b2a0 0x7fe61406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.141+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe61c0899e0 con 0x7fe62813cee0 2023-12-04T05:40:27.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.141+0000 7fe6277fe640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe61406b2a0 0x7fe61406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.141+0000 7fe6277fe640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe61406b2a0 0x7fe61406d760 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe618009ad0 tx=0x7fe618008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:27.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.145+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe61c054850 con 0x7fe62813cee0 2023-12-04T05:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.312+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6280cc720 con 0x7fe62813cee0 2023-12-04T05:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.313+0000 7fe60ffff640 1 -- 172.21.15.111:0/2117571321 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe61c057e70 con 0x7fe62813cee0 2023-12-04T05:40:27.314 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:27.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe61406b2a0 msgr2=0x7fe61406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe61406b2a0 0x7fe61406d760 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe618009ad0 tx=0x7fe618008040 comp rx=0 tx=0).stop 2023-12-04T05:40:27.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 msgr2=0x7fe6281f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 0x7fe6281f2520 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00c430 tx=0x7fe61c00c8f0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 shutdown_connections 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe61406b2a0 0x7fe61406d760 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe62813f640 0x7fe62813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe62813cee0 0x7fe6281f2520 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 --2- 172.21.15.111:0/2117571321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe628072430 0x7fe6280b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 >> 172.21.15.111:0/2117571321 conn(0x7fe6280b0cf0 msgr2=0x7fe6280ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 shutdown_connections 2023-12-04T05:40:27.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.315+0000 7fe62dde5640 1 -- 172.21.15.111:0/2117571321 wait complete. 2023-12-04T05:40:27.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:40:27.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.429+0000 7feed8753640 1 Processor -- start 2023-12-04T05:40:27.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.429+0000 7feed8753640 1 -- start start 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed8753640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feed00d1780 0x7feed00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed8753640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feed00b20a0 0x7feed00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed8753640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00b2ad0 0x7feed00ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed8753640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00cdcb0 con 0x7feed00d1780 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed8753640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00d20a0 con 0x7feed00b20a0 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed8753640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00d39e0 con 0x7feed00b2ad0 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feed00d1780 0x7feed00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feed00d1780 0x7feed00d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60706/0 (socket says 172.21.15.111:60706) 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 -- 172.21.15.111:0/2741980247 learned_addr learned my addr 172.21.15.111:0/2741980247 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed6cf0640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00b2ad0 0x7feed00ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed5cee640 1 --2- 172.21.15.111:0/2741980247 >> v2:172.21.15.192:3300/0 conn(0x7feed00b20a0 0x7feed00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 -- 172.21.15.111:0/2741980247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00b2ad0 msgr2=0x7feed00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00b2ad0 0x7feed00ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 -- 172.21.15.111:0/2741980247 >> v2:172.21.15.192:3300/0 conn(0x7feed00b20a0 msgr2=0x7feed00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 --2- 172.21.15.111:0/2741980247 >> v2:172.21.15.192:3300/0 conn(0x7feed00b20a0 0x7feed00d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed64ef640 1 -- 172.21.15.111:0/2741980247 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00d3cc0 con 0x7feed00d1780 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.430+0000 7feed6cf0640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00b2ad0 0x7feed00ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed64ef640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00d1780 0x7feed00d01e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feed00d0c50 tx=0x7feecc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3724bca7bd293c6d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:27.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed54ed640 1 -- 172.21.15.111:0/2741980247 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0317e0 con 0x7feed00d1780 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed54ed640 1 -- 172.21.15.111:0/2741980247 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc031940 con 0x7feed00d1780 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed54ed640 1 -- 172.21.15.111:0/2741980247 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc031cd0 con 0x7feed00d1780 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 -- 172.21.15.111:0/2741980247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00d1780 msgr2=0x7feed00d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00d1780 0x7feed00d01e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feed00d0c50 tx=0x7feecc02b880 comp rx=0 tx=0).stop 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 -- 172.21.15.111:0/2741980247 shutdown_connections 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00b2ad0 0x7feed00ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 --2- 172.21.15.111:0/2741980247 >> v2:172.21.15.192:3300/0 conn(0x7feed00b20a0 0x7feed00d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 --2- 172.21.15.111:0/2741980247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00d1780 0x7feed00d01e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.431+0000 7feed8753640 1 -- 172.21.15.111:0/2741980247 >> 172.21.15.111:0/2741980247 conn(0x7feed009c4e0 msgr2=0x7feed00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.432+0000 7feed8753640 1 -- 172.21.15.111:0/2741980247 shutdown_connections 2023-12-04T05:40:27.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.432+0000 7feed8753640 1 -- 172.21.15.111:0/2741980247 wait complete. 2023-12-04T05:40:27.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 Processor -- start 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 -- start start 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 0x7feed006cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed00b2ad0 0x7feed0158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00d1780 0x7feed0160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed015f780 con 0x7feed00b20a0 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00ca310 con 0x7feed00b2ad0 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed8753640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed006d0b0 con 0x7feed00d1780 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed64ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 0x7feed006cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed64ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 0x7feed006cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60718/0 (socket says 172.21.15.111:60718) 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.433+0000 7feed64ef640 1 -- 172.21.15.111:0/3597450118 learned_addr learned my addr 172.21.15.111:0/3597450118 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed6cf0640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00d1780 0x7feed0160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed5cee640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed00b2ad0 0x7feed0158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed64ef640 1 -- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00d1780 msgr2=0x7feed0160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed64ef640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00d1780 0x7feed0160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed64ef640 1 -- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed00b2ad0 msgr2=0x7feed0158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed64ef640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed00b2ad0 0x7feed0158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed64ef640 1 -- 172.21.15.111:0/3597450118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0094d0 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed64ef640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 0x7feed006cb70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feecc02b8c0 tx=0x7feecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc032880 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc032a20 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc032dd0 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed015f9d0 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.434+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed00ca580 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.435+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed00693d0 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.436+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7feecc0453f0 con 0x7feed00b20a0 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.436+0000 7feec6ffd640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feeb406b2a0 0x7feeb406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:27.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.436+0000 7feed5cee640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feeb406b2a0 0x7feeb406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:27.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.436+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feecc0abcd0 con 0x7feed00b20a0 2023-12-04T05:40:27.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.436+0000 7feed5cee640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feeb406b2a0 0x7feeb406d760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feec0006fd0 tx=0x7feec0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:27.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.440+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feecc076d20 con 0x7feed00b20a0 2023-12-04T05:40:27.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.621+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feed00d40b0 con 0x7feed00b20a0 2023-12-04T05:40:27.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.621+0000 7feec6ffd640 1 -- 172.21.15.111:0/3597450118 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7feecc07a340 con 0x7feed00b20a0 2023-12-04T05:40:27.623 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feeb406b2a0 msgr2=0x7feeb406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feeb406b2a0 0x7feeb406d760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feec0006fd0 tx=0x7feec0008040 comp rx=0 tx=0).stop 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 msgr2=0x7feed006cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 0x7feed006cb70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feecc02b8c0 tx=0x7feecc0046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 shutdown_connections 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7feeb406b2a0 0x7feeb406d760 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed00d1780 0x7feed0160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed00b2ad0 0x7feed0158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 --2- 172.21.15.111:0/3597450118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feed00b20a0 0x7feed006cb70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 >> 172.21.15.111:0/3597450118 conn(0x7feed009c4e0 msgr2=0x7feed00d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:27.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 shutdown_connections 2023-12-04T05:40:27.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:27.623+0000 7feed8753640 1 -- 172.21.15.111:0/3597450118 wait complete. 2023-12-04T05:40:27.635 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-12-04T05:40:27.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:27.635 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:27.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[103842]: fuse finished with error 0 and tester_r 0 2023-12-04T05:40:27.662 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:33.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:40:33.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:33.764 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-04T05:40:33.783 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:40:33.783 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:33.802 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:33.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.930+0000 7f0155361640 1 Processor -- start 2023-12-04T05:40:33.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.931+0000 7f0155361640 1 -- start start 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.931+0000 7f0155361640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0150159f90 0x7f01500ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.931+0000 7f0155361640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f015006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.931+0000 7f0155361640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01500ceae0 con 0x7f01500aed40 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.931+0000 7f0155361640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01500b2890 con 0x7f0150159f90 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.931+0000 7f0155361640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500cedc0 con 0x7f01500ce620 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f01500aed40 0x7f015015db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52110/0 (socket says 172.21.15.111:52110) 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014f7fe640 1 -- 172.21.15.111:0/1355695830 learned_addr learned my addr 172.21.15.111:0/1355695830 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014effd640 1 --2- 172.21.15.111:0/1355695830 >> v2:172.21.15.192:3300/0 conn(0x7f0150159f90 0x7f01500ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f01467fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f015006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269092694 0 0) 0x7f01500ceae0 con 0x7f01500aed40 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f013c003520 con 0x7f01500aed40 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893037077 0 0) 0x7f013c003520 con 0x7f01500aed40 2023-12-04T05:40:33.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 msgr2=0x7f015006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f015006b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 >> v2:172.21.15.192:3300/0 conn(0x7f0150159f90 msgr2=0x7f01500ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 --2- 172.21.15.111:0/1355695830 >> v2:172.21.15.192:3300/0 conn(0x7f0150159f90 0x7f01500ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500cf0a0 con 0x7f01500aed40 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014effd640 1 --2- 172.21.15.111:0/1355695830 >> v2:172.21.15.192:3300/0 conn(0x7f0150159f90 0x7f01500ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f01480019f0 con 0x7f01500aed40 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f01467fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f015006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 >> v1:172.21.15.111:6789/0 conn(0x7f01500aed40 legacy=0x7f015015db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f013c003970 0x7f013c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c0047c0 0x7f013c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f013c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500b2890 con 0x7f013c003970 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500cedc0 con 0x7f013c0047c0 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c00b980 con 0x7f01500ce620 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f014effd640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f013c003970 0x7f013c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.932+0000 7f01467fc640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c0047c0 0x7f013c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014f7fe640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f013c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014effd640 1 -- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 msgr2=0x7f013c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014effd640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01500ce620 0x7f013c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014effd640 1 -- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c0047c0 msgr2=0x7f013c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014effd640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c0047c0 0x7f013c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014effd640 1 -- 172.21.15.111:0/1355695830 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138009400 con 0x7f013c003970 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.933+0000 7f014effd640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f013c003970 0x7f013c003d90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0140009c90 tx=0x7f013c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0cc7e5afff4998 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014002c980 con 0x7f013c003970 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014002cae0 con 0x7f013c003970 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f014e7fc640 1 -- 172.21.15.111:0/1355695830 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014002ce90 con 0x7f013c003970 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f0155361640 1 -- 172.21.15.111:0/1355695830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f013c003970 msgr2=0x7f013c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f0155361640 1 --2- 172.21.15.111:0/1355695830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f013c003970 0x7f013c003d90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0140009c90 tx=0x7f013c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f014f7fe640 1 -- 172.21.15.111:0/1355695830 reap_dead start 2023-12-04T05:40:33.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f0155361640 1 -- 172.21.15.111:0/1355695830 shutdown_connections 2023-12-04T05:40:33.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f0155361640 1 -- 172.21.15.111:0/1355695830 >> 172.21.15.111:0/1355695830 conn(0x7f01500b1b50 msgr2=0x7f0150070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:33.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f0155361640 1 -- 172.21.15.111:0/1355695830 shutdown_connections 2023-12-04T05:40:33.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.934+0000 7f0155361640 1 -- 172.21.15.111:0/1355695830 wait complete. 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.935+0000 7f0155361640 1 Processor -- start 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 -- start start 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c003970 0x7f0150143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f015015a8d0 0x7f0150145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 0x7f0150158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015015add0 con 0x7f0150145c80 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015015b0b0 con 0x7f013c003970 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f0155361640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015015b390 con 0x7f015015a8d0 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 0x7f0150158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 0x7f0150158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59800/0 (socket says 172.21.15.111:59800) 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.936+0000 7f014f7fe640 1 -- 172.21.15.111:0/2392198977 learned_addr learned my addr 172.21.15.111:0/2392198977 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014effd640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c003970 0x7f0150143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f01467fc640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f015015a8d0 0x7f0150145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014f7fe640 1 -- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f015015a8d0 msgr2=0x7f0150145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014f7fe640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f015015a8d0 0x7f0150145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014f7fe640 1 -- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c003970 msgr2=0x7f0150143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014f7fe640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c003970 0x7f0150143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014f7fe640 1 -- 172.21.15.111:0/2392198977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015015b670 con 0x7f0150145c80 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f01467fc640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f015015a8d0 0x7f0150145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014effd640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c003970 0x7f0150143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f014f7fe640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 0x7f0150158310 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f01500af190 tx=0x7f0148007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800ab90 con 0x7f0150145c80 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014800acf0 con 0x7f0150145c80 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148013630 con 0x7f0150145c80 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015015b950 con 0x7f0150145c80 2023-12-04T05:40:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.937+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0150158850 con 0x7f0150145c80 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.938+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015015bb40 con 0x7f0150145c80 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.939+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0148013790 con 0x7f0150145c80 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.939+0000 7f0147fff640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f013006b2a0 0x7f013006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.939+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f014808a320 con 0x7f0150145c80 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.939+0000 7f014effd640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f013006b2a0 0x7f013006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.939+0000 7f014effd640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f013006b2a0 0x7f013006d760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0140036000 tx=0x7f0140030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:33.943+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0148055110 con 0x7f0150145c80 2023-12-04T05:40:34.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.111+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01500693d0 con 0x7f0150145c80 2023-12-04T05:40:34.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.112+0000 7f0147fff640 1 -- 172.21.15.111:0/2392198977 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f014800ae60 con 0x7f0150145c80 2023-12-04T05:40:34.113 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:34.114 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:34.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f013006b2a0 msgr2=0x7f013006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f013006b2a0 0x7f013006d760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0140036000 tx=0x7f0140030000 comp rx=0 tx=0).stop 2023-12-04T05:40:34.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 msgr2=0x7f0150158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 0x7f0150158310 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f01500af190 tx=0x7f0148007bd0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 shutdown_connections 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f013006b2a0 0x7f013006d760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0150145c80 0x7f0150158310 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f015015a8d0 0x7f0150145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 --2- 172.21.15.111:0/2392198977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f013c003970 0x7f0150143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 >> 172.21.15.111:0/2392198977 conn(0x7f01500b1b50 msgr2=0x7f0150159270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 shutdown_connections 2023-12-04T05:40:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.114+0000 7f0155361640 1 -- 172.21.15.111:0/2392198977 wait complete. 2023-12-04T05:40:34.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:34.127 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:34.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.229+0000 7fc5c2a56640 1 Processor -- start 2023-12-04T05:40:34.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.229+0000 7fc5c2a56640 1 -- start start 2023-12-04T05:40:34.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c2a56640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc5bc13f640 0x7fc5bc13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c2a56640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 0x7fc5bc072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c2a56640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc13dd10 con 0x7fc5bc13cee0 2023-12-04T05:40:34.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c2a56640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc072670 con 0x7fc5bc13f640 2023-12-04T05:40:34.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c2a56640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc13d2c0 con 0x7fc5bc071d20 2023-12-04T05:40:34.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c0ff3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 0x7fc5bc072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c0ff3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 0x7fc5bc072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:55544/0 (socket says 172.21.15.111:55544) 2023-12-04T05:40:34.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/4039969292 learned_addr learned my addr 172.21.15.111:0/4039969292 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:34.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.230+0000 7fc5bb7fe640 1 --2- 172.21.15.111:0/4039969292 >> v2:172.21.15.192:3300/0 conn(0x7fc5bc13f640 0x7fc5bc13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.231+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/4039969292 >> v2:172.21.15.192:3300/0 conn(0x7fc5bc13f640 msgr2=0x7fc5bc13fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:34.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.231+0000 7fc5c0ff3640 1 --2- 172.21.15.111:0/4039969292 >> v2:172.21.15.192:3300/0 conn(0x7fc5bc13f640 0x7fc5bc13fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.231+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/4039969292 >> v1:172.21.15.192:6789/0 conn(0x7fc5bc13cee0 legacy=0x7fc5bc13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.231+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/4039969292 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc1400e0 con 0x7fc5bc071d20 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.231+0000 7fc5c0ff3640 1 --2- 172.21.15.111:0/4039969292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 0x7fc5bc072130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009e50 tx=0x7fc5b402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6577ebc3b288457 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5baffd640 1 -- 172.21.15.111:0/4039969292 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b402c9e0 con 0x7fc5bc071d20 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5baffd640 1 -- 172.21.15.111:0/4039969292 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b402cb40 con 0x7fc5bc071d20 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5baffd640 1 -- 172.21.15.111:0/4039969292 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40343f0 con 0x7fc5bc071d20 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 -- 172.21.15.111:0/4039969292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 msgr2=0x7fc5bc072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/4039969292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 0x7fc5bc072130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009e50 tx=0x7fc5b402be40 comp rx=0 tx=0).stop 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 -- 172.21.15.111:0/4039969292 shutdown_connections 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/4039969292 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc071d20 0x7fc5bc072130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/4039969292 >> v2:172.21.15.192:3300/0 conn(0x7fc5bc13f640 0x7fc5bc13fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 -- 172.21.15.111:0/4039969292 >> 172.21.15.111:0/4039969292 conn(0x7fc5bc0b0cf0 msgr2=0x7fc5bc0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:34.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 -- 172.21.15.111:0/4039969292 shutdown_connections 2023-12-04T05:40:34.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.232+0000 7fc5c2a56640 1 -- 172.21.15.111:0/4039969292 wait complete. 2023-12-04T05:40:34.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.233+0000 7fc5c2a56640 1 Processor -- start 2023-12-04T05:40:34.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 -- start start 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5bc071d20 0x7fc5bc0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc13cee0 0x7fc5bc06c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 0x7fc5bc1363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b4dc0 con 0x7fc5bc13f640 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b3040 con 0x7fc5bc071d20 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c2a56640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b3320 con 0x7fc5bc13cee0 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c0ff3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 0x7fc5bc1363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c0ff3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 0x7fc5bc1363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59802/0 (socket says 172.21.15.111:59802) 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.234+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/3210182467 learned_addr learned my addr 172.21.15.111:0/3210182467 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5bb7fe640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc13cee0 0x7fc5bc06c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5bbfff640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5bc071d20 0x7fc5bc0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc13cee0 msgr2=0x7fc5bc06c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c0ff3640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc13cee0 0x7fc5bc06c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5bc071d20 msgr2=0x7fc5bc0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c0ff3640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5bc071d20 0x7fc5bc0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c0ff3640 1 -- 172.21.15.111:0/3210182467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4009b30 con 0x7fc5bc13f640 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5bb7fe640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc13cee0 0x7fc5bc06c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5bbfff640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5bc071d20 0x7fc5bc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c0ff3640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 0x7fc5bc1363e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009e50 tx=0x7fc5b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4009510 con 0x7fc5bc13f640 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b403e450 con 0x7fc5bc13f640 2023-12-04T05:40:34.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc0b3600 con 0x7fc5bc13f640 2023-12-04T05:40:34.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b403e800 con 0x7fc5bc13f640 2023-12-04T05:40:34.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.235+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0b3bc0 con 0x7fc5bc13f640 2023-12-04T05:40:34.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.236+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc069360 con 0x7fc5bc13f640 2023-12-04T05:40:34.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.237+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc5b403e960 con 0x7fc5bc13f640 2023-12-04T05:40:34.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.237+0000 7fc5b8ff9640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc5a806b2a0 0x7fc5a806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.237+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5b40ac8f0 con 0x7fc5bc13f640 2023-12-04T05:40:34.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.237+0000 7fc5bbfff640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc5a806b2a0 0x7fc5a806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.237+0000 7fc5bbfff640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc5a806b2a0 0x7fc5a806d760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac000f60 tx=0x7fc5ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.241+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5b4077760 con 0x7fc5bc13f640 2023-12-04T05:40:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.410+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5bc070380 con 0x7fc5bc13f640 2023-12-04T05:40:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.410+0000 7fc5b8ff9640 1 -- 172.21.15.111:0/3210182467 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5b407ad80 con 0x7fc5bc13f640 2023-12-04T05:40:34.412 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc5a806b2a0 msgr2=0x7fc5a806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc5a806b2a0 0x7fc5a806d760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac000f60 tx=0x7fc5ac009040 comp rx=0 tx=0).stop 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 msgr2=0x7fc5bc1363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 0x7fc5bc1363e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009e50 tx=0x7fc5b4004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 shutdown_connections 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc5a806b2a0 0x7fc5a806d760 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f640 0x7fc5bc1363e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5bc13cee0 0x7fc5bc06c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 --2- 172.21.15.111:0/3210182467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5bc071d20 0x7fc5bc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.412+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 >> 172.21.15.111:0/3210182467 conn(0x7fc5bc0b0cf0 msgr2=0x7fc5bc0ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:34.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.413+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 shutdown_connections 2023-12-04T05:40:34.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.413+0000 7fc5c2a56640 1 -- 172.21.15.111:0/3210182467 wait complete. 2023-12-04T05:40:34.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:34.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.527+0000 7f4c6da5c640 1 Processor -- start 2023-12-04T05:40:34.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.527+0000 7f4c6da5c640 1 -- start start 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c6da5c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4c680cd040 0x7f4c680cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c6da5c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680d0d80 con 0x7f4c680cda40 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c6da5c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c68141620 con 0x7f4c6815ec20 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c6da5c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6815f3e0 con 0x7f4c680cd040 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c677fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4c680cda40 0x7f4c680cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52120/0 (socket says 172.21.15.111:52120) 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c677fe640 1 -- 172.21.15.111:0/2239120836 learned_addr learned my addr 172.21.15.111:0/2239120836 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893726317 0 0) 0x7f4c680d0d80 con 0x7f4c680cda40 2023-12-04T05:40:34.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c54003520 con 0x7f4c680cda40 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2239120836 >> v2:172.21.15.192:3300/0 conn(0x7f4c680cd040 0x7f4c680cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827799761 0 0) 0x7f4c54003520 con 0x7f4c680cda40 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 >> v2:172.21.15.192:3300/0 conn(0x7f4c680cd040 msgr2=0x7f4c680cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 --2- 172.21.15.111:0/2239120836 >> v2:172.21.15.192:3300/0 conn(0x7f4c680cd040 0x7f4c680cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 >> v1:172.21.15.192:6789/0 conn(0x7f4c6815ec20 legacy=0x7f4c68140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6815f6c0 con 0x7f4c680cda40 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f4c600019f0 con 0x7f4c680cda40 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 >> v1:172.21.15.111:6789/0 conn(0x7f4c680cda40 legacy=0x7f4c680cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c540047c0 0x7f4c54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c54005180 0x7f4c5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6815f3e0 con 0x7f4c54003970 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c0d0 con 0x7f4c540047c0 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c290 con 0x7f4c54005180 2023-12-04T05:40:34.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.528+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c540047c0 0x7f4c54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c677fe640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c54005180 0x7f4c5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c54005180 msgr2=0x7f4c5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c54005180 0x7f4c5400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c540047c0 msgr2=0x7f4c54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c540047c0 0x7f4c54004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2239120836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54003520 con 0x7f4c54003970 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c677fe640 1 -- 172.21.15.111:0/2239120836 reap_dead start 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.529+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ba30 tx=0x7f4c540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571b76143b5ec46 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58032640 con 0x7f4c54003970 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580327a0 con 0x7f4c54003970 2023-12-04T05:40:34.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c667fc640 1 -- 172.21.15.111:0/2239120836 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58032b30 con 0x7f4c54003970 2023-12-04T05:40:34.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2239120836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 msgr2=0x7f4c54003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ba30 tx=0x7f4c540057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2239120836 shutdown_connections 2023-12-04T05:40:34.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2239120836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c54003d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2239120836 >> 172.21.15.111:0/2239120836 conn(0x7f4c68070af0 msgr2=0x7f4c680662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:34.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2239120836 shutdown_connections 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.530+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2239120836 wait complete. 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.531+0000 7f4c6da5c640 1 Processor -- start 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.531+0000 7f4c6da5c640 1 -- start start 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c6da5c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c680ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c6da5c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680cc8e0 0x7f4c6806b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c6da5c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c681399b0 0x7f4c6813cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c6da5c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68145c50 con 0x7f4c54003970 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c6da5c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68154db0 con 0x7f4c681399b0 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c6da5c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680ccd40 con 0x7f4c680cc8e0 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c66ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c680ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c66ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c680ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59814/0 (socket says 172.21.15.111:59814) 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2498251348 learned_addr learned my addr 172.21.15.111:0/2498251348 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680cc8e0 0x7f4c6806b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.532+0000 7f4c677fe640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c681399b0 0x7f4c6813cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680cc8e0 msgr2=0x7f4c6806b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680cc8e0 0x7f4c6806b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c681399b0 msgr2=0x7f4c6813cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c681399b0 0x7f4c6813cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c66ffd640 1 -- 172.21.15.111:0/2498251348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6806bb10 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c677fe640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c681399b0 0x7f4c6813cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c66ffd640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c680ca3e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4c58035040 tx=0x7f4c58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5803baf0 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5803bc50 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680cad60 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58030410 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.533+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68145870 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.534+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68069360 con 0x7f4c54003970 2023-12-04T05:40:34.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.534+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4c58030570 con 0x7f4c54003970 2023-12-04T05:40:34.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.535+0000 7f4c64ff9640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.535+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c580ac550 con 0x7f4c54003970 2023-12-04T05:40:34.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.535+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.535+0000 7f4c5ffff640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000c00 tx=0x7f4c50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.539+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c5807aee0 con 0x7f4c54003970 2023-12-04T05:40:34.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.707+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c680d1700 con 0x7f4c54003970 2023-12-04T05:40:34.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.707+0000 7f4c64ff9640 1 -- 172.21.15.111:0/2498251348 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c58006c30 con 0x7f4c54003970 2023-12-04T05:40:34.709 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 msgr2=0x7f4c4806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000c00 tx=0x7f4c50009040 comp rx=0 tx=0).stop 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 msgr2=0x7f4c680ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c680ca3e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4c58035040 tx=0x7f4c58004730 comp rx=0 tx=0).stop 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 shutdown_connections 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4c4806b2a0 0x7f4c4806d760 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c681399b0 0x7f4c6813cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c680cc8e0 0x7f4c6806b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 --2- 172.21.15.111:0/2498251348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4c54003970 0x7f4c680ca3e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 >> 172.21.15.111:0/2498251348 conn(0x7f4c68070af0 msgr2=0x7f4c6813f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:34.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 shutdown_connections 2023-12-04T05:40:34.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.709+0000 7f4c6da5c640 1 -- 172.21.15.111:0/2498251348 wait complete. 2023-12-04T05:40:34.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:34.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.823+0000 7f97b1aee640 1 Processor -- start 2023-12-04T05:40:34.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.823+0000 7f97b1aee640 1 -- start start 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97b1aee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 0x7f97ac0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97b1aee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97ac161520 0x7f97ac071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97b1aee640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac0b5110 con 0x7f97ac146540 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97b1aee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac161dc0 con 0x7f97ac161520 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97b1aee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b2190 con 0x7f97ac145ee0 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 0x7f97ac0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97abfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97ac161520 0x7f97ac071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97aaffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f97ac146540 0x7f97ac160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:36380/0 (socket says 172.21.15.111:36380) 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 0x7f97ac0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:55600/0 (socket says 172.21.15.111:55600) 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97abfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97ac161520 0x7f97ac071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34504/0 (socket says 172.21.15.111:34504) 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97aaffd640 1 -- 172.21.15.111:0/2895294569 learned_addr learned my addr 172.21.15.111:0/2895294569 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:34.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97ab7fe640 1 -- 172.21.15.111:0/2895294569 >> v2:172.21.15.192:3300/0 conn(0x7f97ac161520 msgr2=0x7f97ac071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:34.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97ab7fe640 1 --2- 172.21.15.111:0/2895294569 >> v2:172.21.15.192:3300/0 conn(0x7f97ac161520 0x7f97ac071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97ab7fe640 1 -- 172.21.15.111:0/2895294569 >> v1:172.21.15.192:6789/0 conn(0x7f97ac146540 legacy=0x7f97ac160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.824+0000 7f97ab7fe640 1 -- 172.21.15.111:0/2895294569 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac161ae0 con 0x7f97ac145ee0 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.825+0000 7f97ab7fe640 1 --2- 172.21.15.111:0/2895294569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 0x7f97ac0ac870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097a0 tx=0x7f979c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c05af2802477e8d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.825+0000 7f97aa7fc640 1 -- 172.21.15.111:0/2895294569 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c031830 con 0x7f97ac145ee0 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.825+0000 7f97aa7fc640 1 -- 172.21.15.111:0/2895294569 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c031990 con 0x7f97ac145ee0 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.825+0000 7f97aa7fc640 1 -- 172.21.15.111:0/2895294569 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c031d20 con 0x7f97ac145ee0 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 -- 172.21.15.111:0/2895294569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 msgr2=0x7f97ac0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2895294569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 0x7f97ac0ac870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097a0 tx=0x7f979c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 -- 172.21.15.111:0/2895294569 shutdown_connections 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2895294569 >> v2:172.21.15.192:3300/0 conn(0x7f97ac161520 0x7f97ac071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2895294569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac145ee0 0x7f97ac0ac870 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 -- 172.21.15.111:0/2895294569 >> 172.21.15.111:0/2895294569 conn(0x7f97ac0b0cf0 msgr2=0x7f97ac0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:34.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 -- 172.21.15.111:0/2895294569 shutdown_connections 2023-12-04T05:40:34.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.826+0000 7f97b1aee640 1 -- 172.21.15.111:0/2895294569 wait complete. 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.827+0000 7f97b1aee640 1 Processor -- start 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.827+0000 7f97b1aee640 1 -- start start 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97b1aee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97ac145ee0 0x7f97ac160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97b1aee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac146540 0x7f97ac13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97b1aee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 0x7f97ac1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97b1aee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac13d040 con 0x7f97ac161520 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97b1aee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac144ed0 con 0x7f97ac145ee0 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97b1aee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac144110 con 0x7f97ac146540 2023-12-04T05:40:34.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 0x7f97ac1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 0x7f97ac1f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59820/0 (socket says 172.21.15.111:59820) 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 -- 172.21.15.111:0/2989208971 learned_addr learned my addr 172.21.15.111:0/2989208971 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 -- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac146540 msgr2=0x7f97ac13cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97aaffd640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac146540 0x7f97ac13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac146540 0x7f97ac13cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 -- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97ac145ee0 msgr2=0x7f97ac160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97ab7fe640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97ac145ee0 0x7f97ac160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97ac145ee0 0x7f97ac160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97abfff640 1 -- 172.21.15.111:0/2989208971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c009400 con 0x7f97ac161520 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.828+0000 7f97ab7fe640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97ac145ee0 0x7f97ac160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.829+0000 7f97abfff640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 0x7f97ac1f6280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f97a400efc0 tx=0x7f97a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.829+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4009ba0 con 0x7f97ac161520 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.829+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400f040 con 0x7f97ac161520 2023-12-04T05:40:34.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.829+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac1443f0 con 0x7f97ac161520 2023-12-04T05:40:34.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.829+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40146c0 con 0x7f97ac161520 2023-12-04T05:40:34.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.829+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1449b0 con 0x7f97ac161520 2023-12-04T05:40:34.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.830+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac144bd0 con 0x7f97ac161520 2023-12-04T05:40:34.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.830+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f97a4014820 con 0x7f97ac161520 2023-12-04T05:40:34.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.831+0000 7f9793fff640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f979806b2a0 0x7f979806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:34.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.831+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97a408a6b0 con 0x7f97ac161520 2023-12-04T05:40:34.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.831+0000 7f97ab7fe640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f979806b2a0 0x7f979806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:34.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.831+0000 7f97ab7fe640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f979806b2a0 0x7f979806d760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f979c036000 tx=0x7f979c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:34.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:34.835+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97a4055520 con 0x7f97ac161520 2023-12-04T05:40:35.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.001+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97ac144250 con 0x7f97ac161520 2023-12-04T05:40:35.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.001+0000 7f9793fff640 1 -- 172.21.15.111:0/2989208971 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97ac144250 con 0x7f97ac161520 2023-12-04T05:40:35.003 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:35.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f979806b2a0 msgr2=0x7f979806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f979806b2a0 0x7f979806d760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f979c036000 tx=0x7f979c008030 comp rx=0 tx=0).stop 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 msgr2=0x7f97ac1f6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 0x7f97ac1f6280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f97a400efc0 tx=0x7f97a400c490 comp rx=0 tx=0).stop 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 shutdown_connections 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f979806b2a0 0x7f979806d760 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac161520 0x7f97ac1f6280 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97ac146540 0x7f97ac13cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 --2- 172.21.15.111:0/2989208971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97ac145ee0 0x7f97ac160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 >> 172.21.15.111:0/2989208971 conn(0x7f97ac0b0cf0 msgr2=0x7f97ac0aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 shutdown_connections 2023-12-04T05:40:35.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.003+0000 7f97b1aee640 1 -- 172.21.15.111:0/2989208971 wait complete. 2023-12-04T05:40:35.015 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:40:35.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.115+0000 7f85d3b33640 1 Processor -- start 2023-12-04T05:40:35.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.115+0000 7f85d3b33640 1 -- start start 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d3b33640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc144430 0x7f85cc15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d3b33640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85cc06c250 0x7f85cc06c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d3b33640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc070bd0 con 0x7f85cc0703f0 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d3b33640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85cc144810 con 0x7f85cc06c250 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d3b33640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc06cc20 con 0x7f85cc144430 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d20d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85cc06c250 0x7f85cc06c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d10ce640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f85cc0703f0 0x7f85cc071c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52128/0 (socket says 172.21.15.111:52128) 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d20d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85cc06c250 0x7f85cc06c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59828/0 (socket says 172.21.15.111:59828) 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d10ce640 1 -- 172.21.15.111:0/2803025330 learned_addr learned my addr 172.21.15.111:0/2803025330 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d18cf640 1 --2- 172.21.15.111:0/2803025330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc144430 0x7f85cc15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d20d0640 1 -- 172.21.15.111:0/2803025330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc144430 msgr2=0x7f85cc15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:35.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d20d0640 1 --2- 172.21.15.111:0/2803025330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc144430 0x7f85cc15f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d20d0640 1 -- 172.21.15.111:0/2803025330 >> v1:172.21.15.111:6789/0 conn(0x7f85cc0703f0 legacy=0x7f85cc071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d08cd640 1 -- 172.21.15.111:0/2803025330 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330352995 0 0) 0x7f85cc070bd0 con 0x7f85cc0703f0 2023-12-04T05:40:35.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.116+0000 7f85d20d0640 1 -- 172.21.15.111:0/2803025330 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc070eb0 con 0x7f85cc06c250 2023-12-04T05:40:35.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.117+0000 7f85d20d0640 1 --2- 172.21.15.111:0/2803025330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc06c250 0x7f85cc06c6e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009d90 tx=0x7f85c402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e552888585bf1b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.117+0000 7f85d08cd640 1 -- 172.21.15.111:0/2803025330 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c402cc50 con 0x7f85cc06c250 2023-12-04T05:40:35.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.117+0000 7f85d08cd640 1 -- 172.21.15.111:0/2803025330 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c402cdb0 con 0x7f85cc06c250 2023-12-04T05:40:35.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.117+0000 7f85d08cd640 1 -- 172.21.15.111:0/2803025330 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4034660 con 0x7f85cc06c250 2023-12-04T05:40:35.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 -- 172.21.15.111:0/2803025330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc06c250 msgr2=0x7f85cc06c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 --2- 172.21.15.111:0/2803025330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc06c250 0x7f85cc06c6e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009d90 tx=0x7f85c402bea0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 -- 172.21.15.111:0/2803025330 shutdown_connections 2023-12-04T05:40:35.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 --2- 172.21.15.111:0/2803025330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc06c250 0x7f85cc06c6e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 --2- 172.21.15.111:0/2803025330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc144430 0x7f85cc15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 -- 172.21.15.111:0/2803025330 >> 172.21.15.111:0/2803025330 conn(0x7f85cc06a4c0 msgr2=0x7f85cc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 -- 172.21.15.111:0/2803025330 shutdown_connections 2023-12-04T05:40:35.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.118+0000 7f85d3b33640 1 -- 172.21.15.111:0/2803025330 wait complete. 2023-12-04T05:40:35.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.119+0000 7f85d3b33640 1 Processor -- start 2023-12-04T05:40:35.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.119+0000 7f85d3b33640 1 -- start start 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85cc06c250 0x7f85cc0c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc0703f0 0x7f85cc13c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 0x7f85cc140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0c48f0 con 0x7f85cc144430 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0c4b00 con 0x7f85cc06c250 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0c4de0 con 0x7f85cc0703f0 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 0x7f85cc140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 0x7f85cc140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59838/0 (socket says 172.21.15.111:59838) 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 -- 172.21.15.111:0/36539065 learned_addr learned my addr 172.21.15.111:0/36539065 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d18cf640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85cc06c250 0x7f85cc0c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 -- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc0703f0 msgr2=0x7f85cc13c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc0703f0 0x7f85cc13c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 -- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85cc06c250 msgr2=0x7f85cc0c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85cc06c250 0x7f85cc0c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 -- 172.21.15.111:0/36539065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4009a70 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d18cf640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85cc06c250 0x7f85cc0c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d20d0640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 0x7f85cc140530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009eb0 tx=0x7f85c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c403f420 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c403f580 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c403f8f0 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc1f26b0 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.120+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc0c50c0 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.121+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85cc1f2bd0 con 0x7f85cc144430 2023-12-04T05:40:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.122+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f85c403fa50 con 0x7f85cc144430 2023-12-04T05:40:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.122+0000 7f85c27fc640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f85b006b2a0 0x7f85b006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.122+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85c40ac8e0 con 0x7f85cc144430 2023-12-04T05:40:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.122+0000 7f85d18cf640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f85b006b2a0 0x7f85b006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.122+0000 7f85d18cf640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f85b006b2a0 0x7f85b006d760 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85c8006fd0 tx=0x7f85c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.127+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85cc1f2bd0 con 0x7f85cc144430 2023-12-04T05:40:35.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.307+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f85cc1f2bd0 con 0x7f85cc144430 2023-12-04T05:40:35.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.307+0000 7f85c27fc640 1 -- 172.21.15.111:0/36539065 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f85cc1f2bd0 con 0x7f85cc144430 2023-12-04T05:40:35.309 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f85b006b2a0 msgr2=0x7f85b006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f85b006b2a0 0x7f85b006d760 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85c8006fd0 tx=0x7f85c8008040 comp rx=0 tx=0).stop 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 msgr2=0x7f85cc140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 0x7f85cc140530 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009eb0 tx=0x7f85c4004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 shutdown_connections 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f85b006b2a0 0x7f85b006d760 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85cc144430 0x7f85cc140530 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85cc0703f0 0x7f85cc13c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 --2- 172.21.15.111:0/36539065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85cc06c250 0x7f85cc0c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 >> 172.21.15.111:0/36539065 conn(0x7f85cc06a4c0 msgr2=0x7f85cc15dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 shutdown_connections 2023-12-04T05:40:35.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.310+0000 7f85d3b33640 1 -- 172.21.15.111:0/36539065 wait complete. 2023-12-04T05:40:35.321 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:40:35.321 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:35.340 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:35.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.441+0000 7f0b3a541640 1 Processor -- start 2023-12-04T05:40:35.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.441+0000 7f0b3a541640 1 -- start start 2023-12-04T05:40:35.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b3a541640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b3415f7b0 0x7f0b34071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b3a541640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b3406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b3a541640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3415fc10 con 0x7f0b34144430 2023-12-04T05:40:35.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b3a541640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b34144810 con 0x7f0b3415f7b0 2023-12-04T05:40:35.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b3a541640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34070bd0 con 0x7f0b340721b0 2023-12-04T05:40:35.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b38ade640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b3406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b337fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b3415f7b0 0x7f0b34071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b33fff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f0b34144430 0x7f0b340703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:36386/0 (socket says 172.21.15.111:36386) 2023-12-04T05:40:35.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b33fff640 1 -- 172.21.15.111:0/1675835083 learned_addr learned my addr 172.21.15.111:0/1675835083 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:35.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b337fe640 1 -- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 msgr2=0x7f0b3406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:35.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b337fe640 1 --2- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b3406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b337fe640 1 -- 172.21.15.111:0/1675835083 >> v1:172.21.15.192:6789/0 conn(0x7f0b34144430 legacy=0x7f0b340703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b337fe640 1 -- 172.21.15.111:0/1675835083 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34070eb0 con 0x7f0b3415f7b0 2023-12-04T05:40:35.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.442+0000 7f0b38ade640 1 --2- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b3406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.443+0000 7f0b337fe640 1 --2- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b34071c70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b280097a0 tx=0x7f0b2802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca219a4b9fae9c26 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.443+0000 7f0b32ffd640 1 -- 172.21.15.111:0/1675835083 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28031830 con 0x7f0b3415f7b0 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.443+0000 7f0b32ffd640 1 -- 172.21.15.111:0/1675835083 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28031990 con 0x7f0b3415f7b0 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.443+0000 7f0b32ffd640 1 -- 172.21.15.111:0/1675835083 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28031d20 con 0x7f0b3415f7b0 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 -- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 msgr2=0x7f0b34071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 --2- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b34071c70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b280097a0 tx=0x7f0b2802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 -- 172.21.15.111:0/1675835083 shutdown_connections 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 --2- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b3406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 --2- 172.21.15.111:0/1675835083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b34071c70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 -- 172.21.15.111:0/1675835083 >> 172.21.15.111:0/1675835083 conn(0x7f0b3406a4c0 msgr2=0x7f0b3406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 -- 172.21.15.111:0/1675835083 shutdown_connections 2023-12-04T05:40:35.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.444+0000 7f0b3a541640 1 -- 172.21.15.111:0/1675835083 wait complete. 2023-12-04T05:40:35.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.445+0000 7f0b3a541640 1 Processor -- start 2023-12-04T05:40:35.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.445+0000 7f0b3a541640 1 -- start start 2023-12-04T05:40:35.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b3a541640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b341311c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b3a541640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 0x7f0b340b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b3a541640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b340cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b3a541640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340c6860 con 0x7f0b34144430 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b3a541640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340b1bf0 con 0x7f0b3415f7b0 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b3a541640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340cb200 con 0x7f0b340721b0 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 0x7f0b340b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 0x7f0b340b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59850/0 (socket says 172.21.15.111:59850) 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 -- 172.21.15.111:0/3369700964 learned_addr learned my addr 172.21.15.111:0/3369700964 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b38ade640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b340cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b33fff640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b341311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 -- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 msgr2=0x7f0b341311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b341311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 -- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 msgr2=0x7f0b340cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b340cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 -- 172.21.15.111:0/3369700964 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c009a70 con 0x7f0b34144430 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.446+0000 7f0b337fe640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 0x7f0b340b16b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0b28000c00 tx=0x7f0b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.447+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2803a830 con 0x7f0b34144430 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.447+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2803a9b0 con 0x7f0b34144430 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.447+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2803ad60 con 0x7f0b34144430 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.447+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28009400 con 0x7f0b34144430 2023-12-04T05:40:35.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.447+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b340c6d50 con 0x7f0b34144430 2023-12-04T05:40:35.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.448+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3413fda0 con 0x7f0b34144430 2023-12-04T05:40:35.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.448+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0b2804d450 con 0x7f0b34144430 2023-12-04T05:40:35.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.449+0000 7f0b30ff9640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b2006b2a0 0x7f0b2006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.449+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b280abd50 con 0x7f0b34144430 2023-12-04T05:40:35.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.449+0000 7f0b33fff640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b2006b2a0 0x7f0b2006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.449+0000 7f0b33fff640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b2006b2a0 0x7f0b2006d760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b24005880 tx=0x7f0b24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.453+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b28076aa0 con 0x7f0b34144430 2023-12-04T05:40:35.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.621+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b3406d100 con 0x7f0b34144430 2023-12-04T05:40:35.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.622+0000 7f0b30ff9640 1 -- 172.21.15.111:0/3369700964 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f0b2807a0c0 con 0x7f0b34144430 2023-12-04T05:40:35.624 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:35.624 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:35.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b2006b2a0 msgr2=0x7f0b2006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b2006b2a0 0x7f0b2006d760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b24005880 tx=0x7f0b24009040 comp rx=0 tx=0).stop 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 msgr2=0x7f0b340b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 0x7f0b340b16b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0b28000c00 tx=0x7f0b280046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 shutdown_connections 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0b2006b2a0 0x7f0b2006d760 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b3415f7b0 0x7f0b340cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b34144430 0x7f0b340b16b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 --2- 172.21.15.111:0/3369700964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b340721b0 0x7f0b341311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 >> 172.21.15.111:0/3369700964 conn(0x7f0b3406a4c0 msgr2=0x7f0b340703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 shutdown_connections 2023-12-04T05:40:35.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.624+0000 7f0b3a541640 1 -- 172.21.15.111:0/3369700964 wait complete. 2023-12-04T05:40:35.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:35.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:35.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.738+0000 7fc8510b2640 1 Processor -- start 2023-12-04T05:40:35.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.738+0000 7fc8510b2640 1 -- start start 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc8510b2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc84c070110 0x7fc84c06f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc8510b2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c0cfd60 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc8510b2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c09a5d0 con 0x7fc84c099a00 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc8510b2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84c09a8b0 con 0x7fc84c070110 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc84a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc84c070110 0x7fc84c06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc84a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc84c070110 0x7fc84c06f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59858/0 (socket says 172.21.15.111:59858) 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc84b59e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc84c0cc3a0 0x7fc84c0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52140/0 (socket says 172.21.15.111:52140) 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc84a59c640 1 -- 172.21.15.111:0/534141487 learned_addr learned my addr 172.21.15.111:0/534141487 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc84a59c640 1 -- 172.21.15.111:0/534141487 >> v1:172.21.15.192:6789/0 conn(0x7fc84c099a00 legacy=0x7fc84c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.739+0000 7fc84a59c640 1 -- 172.21.15.111:0/534141487 >> v1:172.21.15.111:6789/0 conn(0x7fc84c0cc3a0 legacy=0x7fc84c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.740+0000 7fc84a59c640 1 -- 172.21.15.111:0/534141487 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c0cf560 con 0x7fc84c070110 2023-12-04T05:40:35.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.740+0000 7fc849d9b640 1 -- 172.21.15.111:0/534141487 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66397987 0 0) 0x7fc84c0cfd60 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.740+0000 7fc84a59c640 1 --2- 172.21.15.111:0/534141487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c070110 0x7fc84c06f600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098a0 tx=0x7fc84002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0fa3e3db6670ca server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.740+0000 7fc849d9b640 1 -- 172.21.15.111:0/534141487 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840030840 con 0x7fc84c070110 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.740+0000 7fc849d9b640 1 -- 172.21.15.111:0/534141487 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8400309a0 con 0x7fc84c070110 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.740+0000 7fc849d9b640 1 -- 172.21.15.111:0/534141487 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840030d30 con 0x7fc84c070110 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 -- 172.21.15.111:0/534141487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c070110 msgr2=0x7fc84c06f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 --2- 172.21.15.111:0/534141487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c070110 0x7fc84c06f600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098a0 tx=0x7fc84002b980 comp rx=0 tx=0).stop 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 -- 172.21.15.111:0/534141487 shutdown_connections 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 --2- 172.21.15.111:0/534141487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c070110 0x7fc84c06f600 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 -- 172.21.15.111:0/534141487 >> 172.21.15.111:0/534141487 conn(0x7fc84c0b2a70 msgr2=0x7fc84c0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 -- 172.21.15.111:0/534141487 shutdown_connections 2023-12-04T05:40:35.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.741+0000 7fc8510b2640 1 -- 172.21.15.111:0/534141487 wait complete. 2023-12-04T05:40:35.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.742+0000 7fc8510b2640 1 Processor -- start 2023-12-04T05:40:35.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.742+0000 7fc8510b2640 1 -- start start 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc8510b2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c070110 0x7fc84c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc8510b2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc84c099a00 0x7fc84c0c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc8510b2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 0x7fc84c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc8510b2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c5ed0 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc8510b2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c61b0 con 0x7fc84c070110 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc8510b2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c6490 con 0x7fc84c099a00 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc84b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 0x7fc84c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc84b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 0x7fc84c15ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59862/0 (socket says 172.21.15.111:59862) 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc84b59e640 1 -- 172.21.15.111:0/1726350603 learned_addr learned my addr 172.21.15.111:0/1726350603 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc84ad9d640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c070110 0x7fc84c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.743+0000 7fc84a59c640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc84c099a00 0x7fc84c0c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84b59e640 1 -- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc84c099a00 msgr2=0x7fc84c0c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84b59e640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc84c099a00 0x7fc84c0c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84b59e640 1 -- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c070110 msgr2=0x7fc84c1542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84b59e640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c070110 0x7fc84c1542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84b59e640 1 -- 172.21.15.111:0/1726350603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400094d0 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84ad9d640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c070110 0x7fc84c1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc84b59e640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 0x7fc84c15ada0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c450 tx=0x7fc84400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440074e0 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc844007660 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c0c6770 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844004a20 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.744+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c1547f0 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.745+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c069360 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.745+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc844004b80 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.746+0000 7fc8337fe640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83806b2a0 0x7fc83806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.746+0000 7fc84ad9d640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83806b2a0 0x7fc83806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:35.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.746+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc844088d20 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.746+0000 7fc84ad9d640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83806b2a0 0x7fc83806d760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc84c154b70 tx=0x7fc83c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:35.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.750+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc844053c50 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.915+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc84c09ee10 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.915+0000 7fc8337fe640 1 -- 172.21.15.111:0/1726350603 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc844018020 con 0x7fc84c0cc3a0 2023-12-04T05:40:35.917 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83806b2a0 msgr2=0x7fc83806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83806b2a0 0x7fc83806d760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc84c154b70 tx=0x7fc83c009040 comp rx=0 tx=0).stop 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 msgr2=0x7fc84c15ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 0x7fc84c15ada0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c450 tx=0x7fc84400c910 comp rx=0 tx=0).stop 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 shutdown_connections 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83806b2a0 0x7fc83806d760 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc84c0cc3a0 0x7fc84c15ada0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc84c099a00 0x7fc84c0c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 --2- 172.21.15.111:0/1726350603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc84c070110 0x7fc84c1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:35.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 >> 172.21.15.111:0/1726350603 conn(0x7fc84c0b2a70 msgr2=0x7fc84c15cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:35.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.917+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 shutdown_connections 2023-12-04T05:40:35.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:35.918+0000 7fc8510b2640 1 -- 172.21.15.111:0/1726350603 wait complete. 2023-12-04T05:40:35.930 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:36.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.031+0000 7f05ea77c640 1 Processor -- start 2023-12-04T05:40:36.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.031+0000 7f05ea77c640 1 -- start start 2023-12-04T05:40:36.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05ea77c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05e413cee0 0x7f05e413d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05ea77c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413f640 0x7f05e413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05ea77c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e413d2c0 con 0x7f05e4071d20 2023-12-04T05:40:36.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05ea77c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e41400e0 con 0x7f05e413cee0 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05ea77c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41403c0 con 0x7f05e413f640 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05e3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05e413cee0 0x7f05e413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05e3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05e413cee0 0x7f05e413d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59866/0 (socket says 172.21.15.111:59866) 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.032+0000 7f05e3fff640 1 -- 172.21.15.111:0/3573465142 learned_addr learned my addr 172.21.15.111:0/3573465142 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.033+0000 7f05e37fe640 1 --2- 172.21.15.111:0/3573465142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413f640 0x7f05e413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.033+0000 7f05e3fff640 1 -- 172.21.15.111:0/3573465142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413f640 msgr2=0x7f05e413fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.033+0000 7f05e3fff640 1 --2- 172.21.15.111:0/3573465142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413f640 0x7f05e413fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.033+0000 7f05e3fff640 1 -- 172.21.15.111:0/3573465142 >> v1:172.21.15.192:6789/0 conn(0x7f05e4071d20 legacy=0x7f05e40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.033+0000 7f05e3fff640 1 -- 172.21.15.111:0/3573465142 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40cdb50 con 0x7f05e413cee0 2023-12-04T05:40:36.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.033+0000 7f05e3fff640 1 --2- 172.21.15.111:0/3573465142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e413cee0 0x7f05e413d600 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009870 tx=0x7f05d402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e1ddbb33c31be0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05e2ffd640 1 -- 172.21.15.111:0/3573465142 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40317c0 con 0x7f05e413cee0 2023-12-04T05:40:36.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05e2ffd640 1 -- 172.21.15.111:0/3573465142 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4031920 con 0x7f05e413cee0 2023-12-04T05:40:36.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05e2ffd640 1 -- 172.21.15.111:0/3573465142 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4031cb0 con 0x7f05e413cee0 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 -- 172.21.15.111:0/3573465142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e413cee0 msgr2=0x7f05e413d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3573465142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e413cee0 0x7f05e413d600 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009870 tx=0x7f05d402b860 comp rx=0 tx=0).stop 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 -- 172.21.15.111:0/3573465142 shutdown_connections 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3573465142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413f640 0x7f05e413fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3573465142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e413cee0 0x7f05e413d600 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 -- 172.21.15.111:0/3573465142 >> 172.21.15.111:0/3573465142 conn(0x7f05e40b0cf0 msgr2=0x7f05e40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 -- 172.21.15.111:0/3573465142 shutdown_connections 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.034+0000 7f05ea77c640 1 -- 172.21.15.111:0/3573465142 wait complete. 2023-12-04T05:40:36.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.035+0000 7f05ea77c640 1 Processor -- start 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.035+0000 7f05ea77c640 1 -- start start 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05ea77c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 0x7f05e4141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05ea77c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413cee0 0x7f05e4150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05ea77c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05e413f640 0x7f05e412e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05ea77c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41421f0 con 0x7f05e4071d20 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05ea77c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4141f10 con 0x7f05e413f640 2023-12-04T05:40:36.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05ea77c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4132520 con 0x7f05e413cee0 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 0x7f05e4141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 0x7f05e4141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59880/0 (socket says 172.21.15.111:59880) 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 -- 172.21.15.111:0/3520907534 learned_addr learned my addr 172.21.15.111:0/3520907534 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e8d19640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05e413f640 0x7f05e412e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e37fe640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413cee0 0x7f05e4150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 -- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413cee0 msgr2=0x7f05e4150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413cee0 0x7f05e4150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 -- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05e413f640 msgr2=0x7f05e412e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05e413f640 0x7f05e412e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.036+0000 7f05e3fff640 1 -- 172.21.15.111:0/3520907534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40094d0 con 0x7f05e4071d20 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05e8d19640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05e413f640 0x7f05e412e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05e3fff640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 0x7f05e4141930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05d402b8a0 tx=0x7f05d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4032860 con 0x7f05e4071d20 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4032a00 con 0x7f05e4071d20 2023-12-04T05:40:36.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e4132800 con 0x7f05e4071d20 2023-12-04T05:40:36.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4032db0 con 0x7f05e4071d20 2023-12-04T05:40:36.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.037+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e4150c20 con 0x7f05e4071d20 2023-12-04T05:40:36.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.038+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e4132d20 con 0x7f05e4071d20 2023-12-04T05:40:36.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.038+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f05d404e450 con 0x7f05e4071d20 2023-12-04T05:40:36.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.038+0000 7f05e0ff9640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05d006b2a0 0x7f05d006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.039+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d40abde0 con 0x7f05e4071d20 2023-12-04T05:40:36.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.039+0000 7f05e37fe640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05d006b2a0 0x7f05d006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.039+0000 7f05e37fe640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05d006b2a0 0x7f05d006d760 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f05d8006fd0 tx=0x7f05d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.044+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05d4076d10 con 0x7f05e4071d20 2023-12-04T05:40:36.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.207+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05e40cc730 con 0x7f05e4071d20 2023-12-04T05:40:36.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.208+0000 7f05e0ff9640 1 -- 172.21.15.111:0/3520907534 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05d407a330 con 0x7f05e4071d20 2023-12-04T05:40:36.209 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:36.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05d006b2a0 msgr2=0x7f05d006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05d006b2a0 0x7f05d006d760 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f05d8006fd0 tx=0x7f05d8008040 comp rx=0 tx=0).stop 2023-12-04T05:40:36.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 msgr2=0x7f05e4141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 0x7f05e4141930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05d402b8a0 tx=0x7f05d40046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 shutdown_connections 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f05d006b2a0 0x7f05d006d760 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05e413f640 0x7f05e412e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05e413cee0 0x7f05e4150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 --2- 172.21.15.111:0/3520907534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05e4071d20 0x7f05e4141930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 >> 172.21.15.111:0/3520907534 conn(0x7f05e40b0cf0 msgr2=0x7f05e413d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 shutdown_connections 2023-12-04T05:40:36.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.210+0000 7f05ea77c640 1 -- 172.21.15.111:0/3520907534 wait complete. 2023-12-04T05:40:36.221 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.323+0000 7f8942701640 1 Processor -- start 2023-12-04T05:40:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.323+0000 7f8942701640 1 -- start start 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f8942701640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c15b0c0 0x7f893c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f8942701640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f893c0ce940 0x7f893c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f8942701640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c0cd320 con 0x7f893c071b40 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f8942701640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c0ced80 con 0x7f893c0ce940 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f8942701640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0cbee0 con 0x7f893c15b0c0 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f893b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f893c0ce940 0x7f893c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f893b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f893c0ce940 0x7f893c0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59882/0 (socket says 172.21.15.111:59882) 2023-12-04T05:40:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f893b7fe640 1 -- 172.21.15.111:0/916448758 learned_addr learned my addr 172.21.15.111:0/916448758 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f893bfff640 1 --2- 172.21.15.111:0/916448758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c15b0c0 0x7f893c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.324+0000 7f893b7fe640 1 -- 172.21.15.111:0/916448758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c15b0c0 msgr2=0x7f893c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893b7fe640 1 --2- 172.21.15.111:0/916448758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c15b0c0 0x7f893c15ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893b7fe640 1 -- 172.21.15.111:0/916448758 >> v1:172.21.15.192:6789/0 conn(0x7f893c071b40 legacy=0x7f893c15b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893b7fe640 1 -- 172.21.15.111:0/916448758 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0cc710 con 0x7f893c0ce940 2023-12-04T05:40:36.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893b7fe640 1 --2- 172.21.15.111:0/916448758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c0ce940 0x7f893c0cb9a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8930009870 tx=0x7f893002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3b9fa901f3e8a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893affd640 1 -- 172.21.15.111:0/916448758 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300317c0 con 0x7f893c0ce940 2023-12-04T05:40:36.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893affd640 1 -- 172.21.15.111:0/916448758 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8930031920 con 0x7f893c0ce940 2023-12-04T05:40:36.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.325+0000 7f893affd640 1 -- 172.21.15.111:0/916448758 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930031cb0 con 0x7f893c0ce940 2023-12-04T05:40:36.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 -- 172.21.15.111:0/916448758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c0ce940 msgr2=0x7f893c0cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 --2- 172.21.15.111:0/916448758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c0ce940 0x7f893c0cb9a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8930009870 tx=0x7f893002b860 comp rx=0 tx=0).stop 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 -- 172.21.15.111:0/916448758 shutdown_connections 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 --2- 172.21.15.111:0/916448758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c0ce940 0x7f893c0cb9a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 --2- 172.21.15.111:0/916448758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c15b0c0 0x7f893c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 -- 172.21.15.111:0/916448758 >> 172.21.15.111:0/916448758 conn(0x7f893c0b0cf0 msgr2=0x7f893c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 -- 172.21.15.111:0/916448758 shutdown_connections 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.326+0000 7f8942701640 1 -- 172.21.15.111:0/916448758 wait complete. 2023-12-04T05:40:36.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.327+0000 7f8942701640 1 Processor -- start 2023-12-04T05:40:36.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.327+0000 7f8942701640 1 -- start start 2023-12-04T05:40:36.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8942701640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f893c071b40 0x7f893c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8942701640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c0ce940 0x7f893c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8942701640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 0x7f893c0a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8942701640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c156950 con 0x7f893c15b0c0 2023-12-04T05:40:36.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8942701640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1f1d50 con 0x7f893c071b40 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8942701640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c156300 con 0x7f893c0ce940 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 0x7f893c0a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 0x7f893c0a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59884/0 (socket says 172.21.15.111:59884) 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 -- 172.21.15.111:0/3762755343 learned_addr learned my addr 172.21.15.111:0/3762755343 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 -- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c0ce940 msgr2=0x7f893c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c0ce940 0x7f893c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 -- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f893c071b40 msgr2=0x7f893c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f893bfff640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f893c071b40 0x7f893c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f893c071b40 0x7f893c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.328+0000 7f8940c9e640 1 -- 172.21.15.111:0/3762755343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89300094d0 con 0x7f893c15b0c0 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.329+0000 7f8940c9e640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 0x7f893c0a8460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f893400c780 tx=0x7f893400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.329+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934004b60 con 0x7f893c15b0c0 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.329+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893400ce90 con 0x7f893c15b0c0 2023-12-04T05:40:36.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.329+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400a850 con 0x7f893c15b0c0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.329+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c1565e0 con 0x7f893c15b0c0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.329+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c15e490 con 0x7f893c15b0c0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.330+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c156ac0 con 0x7f893c15b0c0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.330+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f893400a9b0 con 0x7f893c15b0c0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.330+0000 7f8938ff9640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f892806b2a0 0x7f892806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.331+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8934089b50 con 0x7f893c15b0c0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.331+0000 7f893bfff640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f892806b2a0 0x7f892806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.331+0000 7f893bfff640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f892806b2a0 0x7f892806d760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f892c00b7c0 tx=0x7f892c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.335+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8934054a50 con 0x7f893c15b0c0 2023-12-04T05:40:36.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.503+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f893c15e490 con 0x7f893c15b0c0 2023-12-04T05:40:36.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.504+0000 7f8938ff9640 1 -- 172.21.15.111:0/3762755343 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f893c15e490 con 0x7f893c15b0c0 2023-12-04T05:40:36.504 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:36.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f892806b2a0 msgr2=0x7f892806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f892806b2a0 0x7f892806d760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f892c00b7c0 tx=0x7f892c008040 comp rx=0 tx=0).stop 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 msgr2=0x7f893c0a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 0x7f893c0a8460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f893400c780 tx=0x7f893400cc40 comp rx=0 tx=0).stop 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 shutdown_connections 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f892806b2a0 0x7f892806d760 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f893c15b0c0 0x7f893c0a8460 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f893c0ce940 0x7f893c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 --2- 172.21.15.111:0/3762755343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f893c071b40 0x7f893c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 >> 172.21.15.111:0/3762755343 conn(0x7f893c0b0cf0 msgr2=0x7f893c153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 shutdown_connections 2023-12-04T05:40:36.507 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.506+0000 7f8942701640 1 -- 172.21.15.111:0/3762755343 wait complete. 2023-12-04T05:40:36.517 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:40:36.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.618+0000 7fa67b6cf640 1 Processor -- start 2023-12-04T05:40:36.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa67b6cf640 1 -- start start 2023-12-04T05:40:36.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa67b6cf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa6740d01e0 0x7fa6740ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa67b6cf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740cdcb0 con 0x7fa6740b20a0 2023-12-04T05:40:36.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa67b6cf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740d39e0 con 0x7fa6740d1780 2023-12-04T05:40:36.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa67b6cf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6740d05f0 con 0x7fa6740d01e0 2023-12-04T05:40:36.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa678c6a640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa6740b20a0 0x7fa6740b2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52144/0 (socket says 172.21.15.111:52144) 2023-12-04T05:40:36.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.619+0000 7fa678c6a640 1 -- 172.21.15.111:0/1163299448 learned_addr learned my addr 172.21.15.111:0/1163299448 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:36.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993038346 0 0) 0x7fa6740cdcb0 con 0x7fa6740b20a0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa660003520 con 0x7fa6740b20a0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1163299448 >> v2:172.21.15.192:3300/0 conn(0x7fa6740d01e0 0x7fa6740ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1734950146 0 0) 0x7fa660003520 con 0x7fa6740b20a0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 >> v2:172.21.15.192:3300/0 conn(0x7fa6740d01e0 msgr2=0x7fa6740ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 --2- 172.21.15.111:0/1163299448 >> v2:172.21.15.192:3300/0 conn(0x7fa6740d01e0 0x7fa6740ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 >> v1:172.21.15.192:6789/0 conn(0x7fa6740d1780 legacy=0x7fa6740d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740d3cc0 con 0x7fa6740b20a0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fa664001670 con 0x7fa6740b20a0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 >> v1:172.21.15.111:6789/0 conn(0x7fa6740b20a0 legacy=0x7fa6740b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa660003970 0x7fa660003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600047c0 0x7fa660004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6600052a0 0x7fa66000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740d05f0 con 0x7fa6600052a0 2023-12-04T05:40:36.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66000c170 con 0x7fa6600047c0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66000c330 con 0x7fa660003970 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6600052a0 0x7fa66000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa678c6a640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600047c0 0x7fa660004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa67946b640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa660003970 0x7fa660003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 -- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa660003970 msgr2=0x7fa660003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa660003970 0x7fa660003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 -- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600047c0 msgr2=0x7fa660004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600047c0 0x7fa660004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa679c6c640 1 -- 172.21.15.111:0/1163299448 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa660003520 con 0x7fa6600052a0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.620+0000 7fa67946b640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa660003970 0x7fa660003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.621+0000 7fa679c6c640 1 -- 172.21.15.111:0/1163299448 reap_dead start 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.621+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6600052a0 0x7fa66000b980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00ba00 tx=0x7fa6600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a601de7c7166bcf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.621+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c004020 con 0x7fa6600052a0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.621+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c004180 con 0x7fa6600052a0 2023-12-04T05:40:36.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.621+0000 7fa66bfff640 1 -- 172.21.15.111:0/1163299448 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c003610 con 0x7fa6600052a0 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1163299448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6600052a0 msgr2=0x7fa66000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6600052a0 0x7fa66000b980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00ba00 tx=0x7fa6600057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1163299448 shutdown_connections 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1163299448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6600052a0 0x7fa66000b980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1163299448 >> 172.21.15.111:0/1163299448 conn(0x7fa67409c4e0 msgr2=0x7fa6740d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1163299448 shutdown_connections 2023-12-04T05:40:36.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.622+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1163299448 wait complete. 2023-12-04T05:40:36.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.623+0000 7fa67b6cf640 1 Processor -- start 2023-12-04T05:40:36.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.623+0000 7fa67b6cf640 1 -- start start 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67b6cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600052a0 0x7fa674155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67b6cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6740c7660 0x7fa6740cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67b6cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 0x7fa6740b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67b6cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740b5dc0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67b6cf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740b6010 con 0x7fa6600052a0 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67b6cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740b62f0 con 0x7fa6740c7660 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 0x7fa6740b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 0x7fa6740b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59906/0 (socket says 172.21.15.111:59906) 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 -- 172.21.15.111:0/1367667476 learned_addr learned my addr 172.21.15.111:0/1367667476 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:36.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 -- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6740c7660 msgr2=0x7fa6740cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6740c7660 0x7fa6740cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67946b640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600052a0 0x7fa674155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 -- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600052a0 msgr2=0x7fa674155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600052a0 0x7fa674155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 -- 172.21.15.111:0/1367667476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740b65d0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa67946b640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600052a0 0x7fa674155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa679c6c640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 0x7fa6740b8200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02bf90 tx=0x7fa66c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.624+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c03b440 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.625+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c03b5c0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.625+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c03b770 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.625+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740b68b0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.625+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6740c8c60 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.626+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6740b6dd0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.626+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa66c03b8d0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.626+0000 7fa669ffb640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa65806b2a0 0x7fa65806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:36.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.627+0000 7fa67946b640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa65806b2a0 0x7fa65806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:36.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.627+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa66c0ab790 con 0x7fa6740cb5a0 2023-12-04T05:40:36.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.627+0000 7fa67946b640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa65806b2a0 0x7fa65806d760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa670000f80 tx=0x7fa670009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:36.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.631+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6740b6dd0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.808+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa6740b6dd0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.809+0000 7fa669ffb640 1 -- 172.21.15.111:0/1367667476 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa6740b6dd0 con 0x7fa6740cb5a0 2023-12-04T05:40:36.809 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.810+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa65806b2a0 msgr2=0x7fa65806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.810+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa65806b2a0 0x7fa65806d760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa670000f80 tx=0x7fa670009040 comp rx=0 tx=0).stop 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.810+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 msgr2=0x7fa6740b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.810+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 0x7fa6740b8200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02bf90 tx=0x7fa66c004730 comp rx=0 tx=0).stop 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.810+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 shutdown_connections 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa65806b2a0 0x7fa65806d760 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6740cb5a0 0x7fa6740b8200 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6740c7660 0x7fa6740cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 --2- 172.21.15.111:0/1367667476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6600052a0 0x7fa674155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:36.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 >> 172.21.15.111:0/1367667476 conn(0x7fa67409c4e0 msgr2=0x7fa6740b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:36.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 shutdown_connections 2023-12-04T05:40:36.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:36.811+0000 7fa67b6cf640 1 -- 172.21.15.111:0/1367667476 wait complete. 2023-12-04T05:40:36.822 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-04T05:40:36.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:36.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-04T05:40:36.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[96468]: fuse finished with error 0 and tester_r 0 2023-12-04T05:40:36.854 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:36.886 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:40:36.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:36.886 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-04T05:40:36.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:37.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.043+0000 7ffa02644640 1 Processor -- start 2023-12-04T05:40:37.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.043+0000 7ffa02644640 1 -- start start 2023-12-04T05:40:37.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa02644640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9fc13cee0 0x7ff9fc06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa02644640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff9fc15aba0 0x7ff9fc06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa02644640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc0aea20 con 0x7ff9fc140200 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa02644640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc13d2c0 con 0x7ff9fc13cee0 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa02644640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc15b3a0 con 0x7ff9fc15aba0 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9fc13cee0 0x7ff9fc06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff9fc13cee0 0x7ff9fc06c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59920/0 (socket says 172.21.15.111:59920) 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 -- 172.21.15.111:0/2265938846 learned_addr learned my addr 172.21.15.111:0/2265938846 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa00be1640 1 --2- 172.21.15.111:0/2265938846 >> v2:172.21.15.192:3300/0 conn(0x7ff9fc15aba0 0x7ff9fc06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 -- 172.21.15.111:0/2265938846 >> v2:172.21.15.192:3300/0 conn(0x7ff9fc15aba0 msgr2=0x7ff9fc06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 --2- 172.21.15.111:0/2265938846 >> v2:172.21.15.192:3300/0 conn(0x7ff9fc15aba0 0x7ff9fc06c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 -- 172.21.15.111:0/2265938846 >> v1:172.21.15.192:6789/0 conn(0x7ff9fc140200 legacy=0x7ff9fc15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ff9fbfff640 1 -- 172.21.15.111:0/2265938846 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc15b0c0 con 0x7ff9fc13cee0 2023-12-04T05:40:37.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.044+0000 7ffa00be1640 1 --2- 172.21.15.111:0/2265938846 >> v2:172.21.15.192:3300/0 conn(0x7ff9fc15aba0 0x7ff9fc06c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:37.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.045+0000 7ff9fbfff640 1 --2- 172.21.15.111:0/2265938846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc06c250 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0b0c80 tx=0x7ff9ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d692f6bd3c4f75 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:37.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.045+0000 7ff9faffd640 1 -- 172.21.15.111:0/2265938846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0317e0 con 0x7ff9fc13cee0 2023-12-04T05:40:37.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.045+0000 7ff9faffd640 1 -- 172.21.15.111:0/2265938846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec031940 con 0x7ff9fc13cee0 2023-12-04T05:40:37.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.045+0000 7ff9faffd640 1 -- 172.21.15.111:0/2265938846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec031cd0 con 0x7ff9fc13cee0 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 -- 172.21.15.111:0/2265938846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc13cee0 msgr2=0x7ff9fc06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 --2- 172.21.15.111:0/2265938846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc06c250 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0b0c80 tx=0x7ff9ec02b880 comp rx=0 tx=0).stop 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 -- 172.21.15.111:0/2265938846 shutdown_connections 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 --2- 172.21.15.111:0/2265938846 >> v2:172.21.15.192:3300/0 conn(0x7ff9fc15aba0 0x7ff9fc06c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 --2- 172.21.15.111:0/2265938846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc06c250 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 -- 172.21.15.111:0/2265938846 >> 172.21.15.111:0/2265938846 conn(0x7ff9fc06a4c0 msgr2=0x7ff9fc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 -- 172.21.15.111:0/2265938846 shutdown_connections 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.046+0000 7ffa02644640 1 -- 172.21.15.111:0/2265938846 wait complete. 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.047+0000 7ffa02644640 1 Processor -- start 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.047+0000 7ffa02644640 1 -- start start 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa02644640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa02644640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9fc140200 0x7ff9fc152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa02644640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 0x7ff9fc0703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa02644640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0ade00 con 0x7ff9fc15aba0 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa02644640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0ae0e0 con 0x7ff9fc13cee0 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa02644640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0b2400 con 0x7ff9fc140200 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 0x7ff9fc0703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 0x7ff9fc0703f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59932/0 (socket says 172.21.15.111:59932) 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 -- 172.21.15.111:0/2722792669 learned_addr learned my addr 172.21.15.111:0/2722792669 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ff9fb7fe640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9fc140200 0x7ff9fc152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ff9fbfff640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 -- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9fc140200 msgr2=0x7ff9fc152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9fc140200 0x7ff9fc152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 -- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9fc13cee0 msgr2=0x7ff9fc1f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc1f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 -- 172.21.15.111:0/2722792669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0094d0 con 0x7ff9fc15aba0 2023-12-04T05:40:37.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.048+0000 7ffa00be1640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 0x7ff9fc0703f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cd80 tx=0x7ff9f400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:37.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.049+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400f680 con 0x7ff9fc15aba0 2023-12-04T05:40:37.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.049+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f400f800 con 0x7ff9fc15aba0 2023-12-04T05:40:37.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.049+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0b26e0 con 0x7ff9fc15aba0 2023-12-04T05:40:37.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.049+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400fbb0 con 0x7ff9fc15aba0 2023-12-04T05:40:37.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.049+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc152ff0 con 0x7ff9fc15aba0 2023-12-04T05:40:37.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.050+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc0b2920 con 0x7ff9fc15aba0 2023-12-04T05:40:37.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.050+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff9f400b040 con 0x7ff9fc15aba0 2023-12-04T05:40:37.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.050+0000 7ff9f8ff9640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9e806b2a0 0x7ff9e806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.050+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9f408a640 con 0x7ff9fc15aba0 2023-12-04T05:40:37.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.050+0000 7ff9fbfff640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9e806b2a0 0x7ff9e806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.051+0000 7ff9fbfff640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9e806b2a0 0x7ff9e806d760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:37.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.055+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9fc0b2920 con 0x7ff9fc15aba0 2023-12-04T05:40:37.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.256+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9fc0b2920 con 0x7ff9fc15aba0 2023-12-04T05:40:37.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.257+0000 7ff9f8ff9640 1 -- 172.21.15.111:0/2722792669 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7ff9fc0b2920 con 0x7ff9fc15aba0 2023-12-04T05:40:37.257 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:37.257 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:37.258 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.258+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9e806b2a0 msgr2=0x7ff9e806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.258+0000 7ffa02644640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9e806b2a0 0x7ff9e806d760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec036000 tx=0x7ff9ec03d040 comp rx=0 tx=0).stop 2023-12-04T05:40:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.258+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 msgr2=0x7ff9fc0703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.258+0000 7ffa02644640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 0x7ff9fc0703f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cd80 tx=0x7ff9f400a650 comp rx=0 tx=0).stop 2023-12-04T05:40:37.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 shutdown_connections 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9e806b2a0 0x7ff9e806d760 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9fc15aba0 0x7ff9fc0703f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9fc140200 0x7ff9fc152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 --2- 172.21.15.111:0/2722792669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9fc13cee0 0x7ff9fc1f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 >> 172.21.15.111:0/2722792669 conn(0x7ff9fc06a4c0 msgr2=0x7ff9fc15d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 shutdown_connections 2023-12-04T05:40:37.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.259+0000 7ffa02644640 1 -- 172.21.15.111:0/2722792669 wait complete. 2023-12-04T05:40:37.270 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2023-12-04T05:40:37.386 INFO:teuthology.orchestra.run.smithi111.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":67,"msgr_send_messages":62,"msgr_recv_bytes":114767,"msgr_send_bytes":16294,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.010941511,"msgr_running_send_time":0.001553585,"msgr_running_recv_time":0.012961607,"msgr_running_fast_dispatch_time":0.000375645,"msgr_send_messages_queue_lat":{"avgcount":61,"sum":0.043910198,"avgtime":0.000719839},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":112912,"msgr_send_encrypted_bytes":13472},"AsyncMessenger::Worker-1":{"msgr_recv_messages":23,"msgr_send_messages":26,"msgr_recv_bytes":6876,"msgr_send_bytes":121487,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.008838920,"msgr_running_send_time":0.000999614,"msgr_running_recv_time":0.007364018,"msgr_running_fast_dispatch_time":0.000316109,"msgr_send_messages_queue_lat":{"avgcount":25,"sum":0.032489117,"avgtime":0.001299564},"msgr_handle_ack_lat":{"avgcount":19,"sum":0.000002875,"avgtime":0.000000151},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":97696},"AsyncMessenger::Worker-2":{"msgr_recv_messages":50,"msgr_send_messages":39,"msgr_recv_bytes":14780,"msgr_send_bytes":24641,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.006266482,"msgr_running_send_time":0.000817315,"msgr_running_recv_time":0.004047888,"msgr_running_fast_dispatch_time":0.001284460,"msgr_send_messages_queue_lat":{"avgcount":38,"sum":0.110310336,"avgtime":0.002902903},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000002995,"avgtime":0.000000142},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":32,"sum":0.006489916,"avgtime":0.000202809}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000012517,"avgtime":0.000006258}},"mds":{"request":3,"reply":3,"reply_latency":{"avgcount":3,"sum":0.000647208,"avgtime":0.000215736},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":5,"traverse_hit":5,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":52,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197222,"rdpos":0,"jlat":{"avgcount":5,"sum":0.007542813,"avgtime":0.001508562},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":2,"cap-":1,"rss":41888,"heap":207196},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":2,"sum":0.000406431,"avgtime":0.000203215},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000240777,"avgtime":0.000240777},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":2,"average_load":1,"avg_session_uptime":10,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":221427,"buffer_anon_items":63,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51880,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":14128,"op_resend":37,"op_reply":44,"op_latency":{"avgcount":44,"sum":0.587989707,"avgtime":0.013363402},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":33,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":135,"get_sum":119743,"get_or_fail_fail":0,"get_or_fail_success":135,"take":0,"take_sum":0,"put":135,"put_sum":119743,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":14128,"put":44,"put_sum":14128,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5596555866e0":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2918,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":5,"put_sum":2918,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-12-04T05:40:37.387 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 114767, "msgr_recv_encrypted_bytes": 112912, "msgr_recv_messages": 67, "msgr_running_fast_dispatch_time": 0.000375645, "msgr_running_recv_time": 0.012961607, "msgr_running_send_time": 0.001553585, "msgr_running_total_time": 0.010941511, "msgr_send_bytes": 16294, "msgr_send_encrypted_bytes": 13472, "msgr_send_messages": 62, "msgr_send_messages_queue_lat": { "avgcount": 61, "avgtime": 0.000719839, "sum": 0.043910198 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 19, "avgtime": 1.51e-07, "sum": 2.875e-06 }, "msgr_recv_bytes": 6876, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000316109, "msgr_running_recv_time": 0.007364018, "msgr_running_send_time": 0.000999614, "msgr_running_total_time": 0.00883892, "msgr_send_bytes": 121487, "msgr_send_encrypted_bytes": 97696, "msgr_send_messages": 26, "msgr_send_messages_queue_lat": { "avgcount": 25, "avgtime": 0.001299564, "sum": 0.032489117 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 1.42e-07, "sum": 2.995e-06 }, "msgr_recv_bytes": 14780, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 50, "msgr_running_fast_dispatch_time": 0.00128446, "msgr_running_recv_time": 0.004047888, "msgr_running_send_time": 0.000817315, "msgr_running_total_time": 0.006266482, "msgr_send_bytes": 24641, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 39, "msgr_send_messages_queue_lat": { "avgcount": 38, "avgtime": 0.002902903, "sum": 0.110310336 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 32, "avgtime": 0.000202809, "sum": 0.006489916 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 6.258e-06, "sum": 1.2517e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 52, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 3, "reply_latency": { "avgcount": 3, "avgtime": 0.000215736, "sum": 0.000647208 }, "request": 3, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 5, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 5, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.001508562, "sum": 0.007542813 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197222 }, "mds_mem": { "cap": 1, "cap+": 2, "cap-": 1, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207196, "ino": 13, "ino+": 13, "ino-": 0, "rss": 41888 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 2, "avgtime": 0.000203215, "sum": 0.000406431 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000240777, "sum": 0.000240777 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 10, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 2 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 221427, "buffer_anon_items": 63, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51880, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 33, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.013363402, "sum": 0.587989707 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 37, "op_rmw": 0, "op_send": 44, "op_send_bytes": 14128, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 135, "get_or_fail_fail": 0, "get_or_fail_success": 135, "get_started": 0, "get_sum": 119743, "max": 104857600, "put": 135, "put_sum": 119743, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 14128, "take": 44, "take_sum": 14128, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5596555866e0": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2918, "max": 3758096384, "put": 5, "put_sum": 2918, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-12-04T05:40:37.387 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:40:37.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.488+0000 7f4bc5815640 1 Processor -- start 2023-12-04T05:40:37.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.488+0000 7f4bc5815640 1 -- start start 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bc5815640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bc00d1780 0x7f4bc00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bc5815640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00cdcb0 con 0x7f4bc00b2ad0 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bc5815640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00d39e0 con 0x7f4bc00b20a0 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bc5815640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc00d3cc0 con 0x7f4bc00d1780 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbeffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bc00d1780 0x7f4bc00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbf7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4bc00b2ad0 0x7f4bc00ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52160/0 (socket says 172.21.15.111:52160) 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbf7fe640 1 -- 172.21.15.111:0/1026491731 learned_addr learned my addr 172.21.15.111:0/1026491731 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbeffd640 1 -- 172.21.15.111:0/1026491731 >> v1:172.21.15.192:6789/0 conn(0x7f4bc00b20a0 legacy=0x7f4bc00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbeffd640 1 -- 172.21.15.111:0/1026491731 >> v1:172.21.15.111:6789/0 conn(0x7f4bc00b2ad0 legacy=0x7f4bc00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbeffd640 1 -- 172.21.15.111:0/1026491731 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00cdf50 con 0x7f4bc00d1780 2023-12-04T05:40:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.489+0000 7f4bbe7fc640 1 -- 172.21.15.111:0/1026491731 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579140927 0 0) 0x7f4bc00cdcb0 con 0x7f4bc00b2ad0 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bbeffd640 1 --2- 172.21.15.111:0/1026491731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00d1780 0x7f4bc00d01e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459b495a956dd1a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bbe7fc640 1 -- 172.21.15.111:0/1026491731 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb002e070 con 0x7f4bc00d1780 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bbe7fc640 1 -- 172.21.15.111:0/1026491731 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb00315a0 con 0x7f4bc00d1780 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bbe7fc640 1 -- 172.21.15.111:0/1026491731 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0031930 con 0x7f4bc00d1780 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bc5815640 1 -- 172.21.15.111:0/1026491731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00d1780 msgr2=0x7f4bc00d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bc5815640 1 --2- 172.21.15.111:0/1026491731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00d1780 0x7f4bc00d01e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002b840 comp rx=0 tx=0).stop 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bc5815640 1 -- 172.21.15.111:0/1026491731 shutdown_connections 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bc5815640 1 --2- 172.21.15.111:0/1026491731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00d1780 0x7f4bc00d01e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.490+0000 7f4bc5815640 1 -- 172.21.15.111:0/1026491731 >> 172.21.15.111:0/1026491731 conn(0x7f4bc009c4e0 msgr2=0x7f4bc00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.491+0000 7f4bc5815640 1 -- 172.21.15.111:0/1026491731 shutdown_connections 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.491+0000 7f4bc5815640 1 -- 172.21.15.111:0/1026491731 wait complete. 2023-12-04T05:40:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 Processor -- start 2023-12-04T05:40:37.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 -- start start 2023-12-04T05:40:37.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 0x7f4bc01511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc00b2ad0 0x7f4bc00b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bc00d1780 0x7f4bc00b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b7140 con 0x7f4bc00b20a0 2023-12-04T05:40:37.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b7420 con 0x7f4bc00b2ad0 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bc5815640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b7700 con 0x7f4bc00d1780 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 0x7f4bc01511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 0x7f4bc01511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59958/0 (socket says 172.21.15.111:59958) 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.492+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2861187981 learned_addr learned my addr 172.21.15.111:0/2861187981 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bb7fff640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc00b2ad0 0x7f4bc00b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbf7fe640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bc00d1780 0x7f4bc00b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bc00d1780 msgr2=0x7f4bc00b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbeffd640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bc00d1780 0x7f4bc00b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc00b2ad0 msgr2=0x7f4bc00b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbeffd640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc00b2ad0 0x7f4bc00b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbeffd640 1 -- 172.21.15.111:0/2861187981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009400 con 0x7f4bc00b20a0 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bb7fff640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc00b2ad0 0x7f4bc00b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbeffd640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 0x7f4bc01511d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0034040 tx=0x7f4bb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:37.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0032900 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb0032a60 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0032e10 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00b79e0 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.493+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00b6d10 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.494+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc00b6f00 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.494+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4bb003b3f0 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.495+0000 7f4bbcff9640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ba006b2d0 0x7f4ba006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.495+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bb00abc50 con 0x7f4bc00b20a0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.495+0000 7f4bb7fff640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ba006b2d0 0x7f4ba006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:37.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.495+0000 7f4bb7fff640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ba006b2d0 0x7f4ba006d790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00b4710 tx=0x7f4ba8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:37.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.499+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4bc00b6f00 con 0x7f4bc00b20a0 2023-12-04T05:40:37.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.698+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4bc00b6f00 con 0x7f4bc00b20a0 2023-12-04T05:40:37.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.699+0000 7f4bbcff9640 1 -- 172.21.15.111:0/2861187981 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4bc00b6f00 con 0x7f4bc00b20a0 2023-12-04T05:40:37.700 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ba006b2d0 msgr2=0x7f4ba006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ba006b2d0 0x7f4ba006d790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00b4710 tx=0x7f4ba8009040 comp rx=0 tx=0).stop 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 msgr2=0x7f4bc01511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 0x7f4bc01511d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0034040 tx=0x7f4bb00046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 shutdown_connections 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4ba006b2d0 0x7f4ba006d790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bc00d1780 0x7f4bc00b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc00b2ad0 0x7f4bc00b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 --2- 172.21.15.111:0/2861187981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc00b20a0 0x7f4bc01511d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:37.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 >> 172.21.15.111:0/2861187981 conn(0x7f4bc009c4e0 msgr2=0x7f4bc00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:37.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 shutdown_connections 2023-12-04T05:40:37.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:37.701+0000 7f4bc5815640 1 -- 172.21.15.111:0/2861187981 wait complete. 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:40:37.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:40:37.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:37.713 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-12-04T05:40:37.728 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:40:37.728 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:40:37.728 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:40:37.728 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42080sec preferred_lft 42080sec 2023-12-04T05:40:37.729 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4e:91:07:f7:96:4c brd ff:ff:ff:ff:ff:ff 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::142c:d5ff:fef0:ffc5/64 scope link 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8e:82:5e:fe:4f:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::8c82:5eff:fefe:4fbf/64 scope link 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4e:91:07:f7:96:4c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:37.730 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::4c91:7ff:fef7:964c/64 scope link 2023-12-04T05:40:37.731 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:40:37.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:37.731 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:40:37.794 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-12-04T05:40:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:37.796 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-12-04T05:40:37.851 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-12-04T05:40:37.852 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:40:37.852 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:40:37.852 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:40:37.852 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:40:37.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:37.852 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:40:37.852 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:40:37.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:40:37.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:40:37.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:40:37.853 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:40:37.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:37.948 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:40:37.962 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-04T05:40:37.963 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-04T05:40:37.963 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:40:37.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:37.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-12-04T05:40:38.030 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-04T05:40:38.030 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:40:38.030 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:38.085 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:40:38.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:40:38.085 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:38.139 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:40:38.207 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:40:38.263 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:40:38.264 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:38.265 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:38.266 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:38.266 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:38.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:38.268 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:40:38.318 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:40:38.318 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-04T05:40:38.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:40:38.387 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:40:38.443 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:40:38.444 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:38.445 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:40:38.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:38.446 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:40:38.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2023-12-04T05:40:38.446+0000 7f395d23e480 -1 init, newargv = 0x555f1917b460 newargc=15 2023-12-04T05:40:38.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[105312]: starting ceph client 2023-12-04T05:40:38.450 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[105312]: starting fuse 2023-12-04T05:40:38.462 INFO:teuthology.orchestra.run.smithi111.stdout:41 2023-12-04T05:40:38.462 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:40:38.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:40:38.462 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-12-04T05:40:38.462 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-12-04T05:40:38.462 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:40:38.463 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2023-12-04T05:40:38.464 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-12-04T05:40:38.581 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.105312.asok 2023-12-04T05:40:38.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:40:38.592 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105312.asok 2023-12-04T05:40:38.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:38.592 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105312.asok status 2023-12-04T05:40:38.726 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "105312", 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-12-04T05:40:38.727 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5625, 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 5625 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 4187360009 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:40:38.728 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 4187360009 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.5625 192.168.0.1:0/4187360009", 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/4187360009", 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 14, 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 33, 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-12-04T05:40:38.729 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:40:38.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:38.735 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-12-04T05:40:38.735 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:38.750 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-12-04T05:40:38.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:38.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:38.751 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:38.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2023-12-04T05:40:38.971 INFO:teuthology.orchestra.run.smithi111.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":67,"msgr_send_messages":62,"msgr_recv_bytes":114767,"msgr_send_bytes":16294,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.011039817,"msgr_running_send_time":0.001581124,"msgr_running_recv_time":0.012974333,"msgr_running_fast_dispatch_time":0.000375645,"msgr_send_messages_queue_lat":{"avgcount":61,"sum":0.043910198,"avgtime":0.000719839},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":112912,"msgr_send_encrypted_bytes":13472},"AsyncMessenger::Worker-1":{"msgr_recv_messages":26,"msgr_send_messages":29,"msgr_recv_bytes":7704,"msgr_send_bytes":122329,"msgr_created_connections":9,"msgr_active_connections":4,"msgr_running_total_time":0.009232429,"msgr_running_send_time":0.001068153,"msgr_running_recv_time":0.007699695,"msgr_running_fast_dispatch_time":0.000316109,"msgr_send_messages_queue_lat":{"avgcount":28,"sum":0.032515784,"avgtime":0.001161278},"msgr_handle_ack_lat":{"avgcount":25,"sum":0.000003714,"avgtime":0.000000148},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":97696},"AsyncMessenger::Worker-2":{"msgr_recv_messages":51,"msgr_send_messages":40,"msgr_recv_bytes":15026,"msgr_send_bytes":25393,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.006328787,"msgr_running_send_time":0.000838077,"msgr_running_recv_time":0.004063633,"msgr_running_fast_dispatch_time":0.001304913,"msgr_send_messages_queue_lat":{"avgcount":39,"sum":0.110328044,"avgtime":0.002828924},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000002995,"avgtime":0.000000142},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":34,"sum":0.006544391,"avgtime":0.000192482}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000012517,"avgtime":0.000006258}},"mds":{"request":4,"reply":4,"reply_latency":{"avgcount":4,"sum":0.000806800,"avgtime":0.000201700},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":7,"traverse_hit":7,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":52,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197663,"rdpos":0,"jlat":{"avgcount":6,"sum":0.008913540,"avgtime":0.001485590},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":42016,"heap":207196},"mds_server":{"dispatch_client_request":4,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":10,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":3,"sum":0.000566023,"avgtime":0.000188674},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000240777,"avgtime":0.000240777},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":3,"average_load":3,"avg_session_uptime":10,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":221427,"buffer_anon_items":63,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51880,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":45,"op_send_bytes":14569,"op_resend":37,"op_reply":45,"op_latency":{"avgcount":45,"sum":0.588989689,"avgtime":0.013088659},"op_inflight":0,"oplen_avg":{"avgcount":45,"sum":69},"op":45,"op_r":0,"op_w":45,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":6,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":33,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":139,"get_sum":120457,"get_or_fail_fail":0,"get_or_fail_success":139,"take":0,"take_sum":0,"put":139,"put_sum":120457,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":14569,"put":45,"put_sum":14569,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":45,"put":45,"put_sum":45,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5596555866e0":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":3359,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":6,"put_sum":3359,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-12-04T05:40:38.972 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 114767, "msgr_recv_encrypted_bytes": 112912, "msgr_recv_messages": 67, "msgr_running_fast_dispatch_time": 0.000375645, "msgr_running_recv_time": 0.012974333, "msgr_running_send_time": 0.001581124, "msgr_running_total_time": 0.011039817, "msgr_send_bytes": 16294, "msgr_send_encrypted_bytes": 13472, "msgr_send_messages": 62, "msgr_send_messages_queue_lat": { "avgcount": 61, "avgtime": 0.000719839, "sum": 0.043910198 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 25, "avgtime": 1.48e-07, "sum": 3.714e-06 }, "msgr_recv_bytes": 7704, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 26, "msgr_running_fast_dispatch_time": 0.000316109, "msgr_running_recv_time": 0.007699695, "msgr_running_send_time": 0.001068153, "msgr_running_total_time": 0.009232429, "msgr_send_bytes": 122329, "msgr_send_encrypted_bytes": 97696, "msgr_send_messages": 29, "msgr_send_messages_queue_lat": { "avgcount": 28, "avgtime": 0.001161278, "sum": 0.032515784 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 1.42e-07, "sum": 2.995e-06 }, "msgr_recv_bytes": 15026, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 51, "msgr_running_fast_dispatch_time": 0.001304913, "msgr_running_recv_time": 0.004063633, "msgr_running_send_time": 0.000838077, "msgr_running_total_time": 0.006328787, "msgr_send_bytes": 25393, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 40, "msgr_send_messages_queue_lat": { "avgcount": 39, "avgtime": 0.002828924, "sum": 0.110328044 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 34, "avgtime": 0.000192482, "sum": 0.006544391 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 6.258e-06, "sum": 1.2517e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 52, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 4, "reply_latency": { "avgcount": 4, "avgtime": 0.0002017, "sum": 0.0008068 }, "request": 4, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 7, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 7, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 6, "avgtime": 0.00148559, "sum": 0.00891354 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197663 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207196, "ino": 13, "ino+": 13, "ino-": 0, "rss": 42016 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 4, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 10, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 3, "avgtime": 0.000188674, "sum": 0.000566023 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000240777, "sum": 0.000240777 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 3, "avg_session_uptime": 10, "mdthresh_evicted": 0, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 3 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 221427, "buffer_anon_items": 63, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51880, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 33, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 45, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 45, "avgtime": 0.013088659, "sum": 0.588989689 }, "op_pg": 0, "op_r": 0, "op_reply": 45, "op_resend": 37, "op_rmw": 0, "op_send": 45, "op_send_bytes": 14569, "op_w": 45, "oplen_avg": { "avgcount": 45, "sum": 69 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 6, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 139, "get_or_fail_fail": 0, "get_or_fail_success": 139, "get_started": 0, "get_sum": 120457, "max": 104857600, "put": 139, "put_sum": 120457, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 45, "put_sum": 14569, "take": 45, "take_sum": 14569, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 45, "put_sum": 45, "take": 45, "take_sum": 45, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5596555866e0": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 3359, "max": 3758096384, "put": 6, "put_sum": 3359, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-12-04T05:40:38.973 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:38.973 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:38.989 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:39.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.129+0000 7fc19171d640 1 Processor -- start 2023-12-04T05:40:39.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.129+0000 7fc19171d640 1 -- start start 2023-12-04T05:40:39.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.129+0000 7fc19171d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc18c0d1780 0x7fc18c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.129+0000 7fc19171d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c0cdcb0 con 0x7fc18c0b20a0 2023-12-04T05:40:39.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.129+0000 7fc19171d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c0d39e0 con 0x7fc18c0b2ad0 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.129+0000 7fc19171d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18c0d3cc0 con 0x7fc18c0d1780 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc18c0d1780 0x7fc18c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18a7fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc18c0b20a0 0x7fc18c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52166/0 (socket says 172.21.15.111:52166) 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18a7fc640 1 -- 172.21.15.111:0/995415309 learned_addr learned my addr 172.21.15.111:0/995415309 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc18c0d1780 0x7fc18c0d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59960/0 (socket says 172.21.15.111:59960) 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18affd640 1 -- 172.21.15.111:0/995415309 >> v1:172.21.15.192:6789/0 conn(0x7fc18c0b2ad0 legacy=0x7fc18c0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18affd640 1 -- 172.21.15.111:0/995415309 >> v1:172.21.15.111:6789/0 conn(0x7fc18c0b20a0 legacy=0x7fc18c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc18affd640 1 -- 172.21.15.111:0/995415309 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0cdf50 con 0x7fc18c0d1780 2023-12-04T05:40:39.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.130+0000 7fc189ffb640 1 -- 172.21.15.111:0/995415309 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206789263 0 0) 0x7fc18c0cdcb0 con 0x7fc18c0b20a0 2023-12-04T05:40:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc18affd640 1 --2- 172.21.15.111:0/995415309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c0d01e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0097a0 tx=0x7fc17c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890b3e3d5e1bce65 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc189ffb640 1 -- 172.21.15.111:0/995415309 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c031830 con 0x7fc18c0d1780 2023-12-04T05:40:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc189ffb640 1 -- 172.21.15.111:0/995415309 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c031990 con 0x7fc18c0d1780 2023-12-04T05:40:39.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc189ffb640 1 -- 172.21.15.111:0/995415309 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c031d20 con 0x7fc18c0d1780 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 -- 172.21.15.111:0/995415309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 msgr2=0x7fc18c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 --2- 172.21.15.111:0/995415309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c0d01e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0097a0 tx=0x7fc17c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 -- 172.21.15.111:0/995415309 shutdown_connections 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 --2- 172.21.15.111:0/995415309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c0d01e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 -- 172.21.15.111:0/995415309 >> 172.21.15.111:0/995415309 conn(0x7fc18c09c4e0 msgr2=0x7fc18c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 -- 172.21.15.111:0/995415309 shutdown_connections 2023-12-04T05:40:39.133 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.131+0000 7fc19171d640 1 -- 172.21.15.111:0/995415309 wait complete. 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 Processor -- start 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 -- start start 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0b20a0 0x7fc18c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc18c0b2ad0 0x7fc18c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c151360 con 0x7fc18c0d1780 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c151640 con 0x7fc18c0b20a0 2023-12-04T05:40:39.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc19171d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c151920 con 0x7fc18c0b2ad0 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc18b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc18b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c1f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59964/0 (socket says 172.21.15.111:59964) 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.133+0000 7fc18b7fe640 1 -- 172.21.15.111:0/4136136002 learned_addr learned my addr 172.21.15.111:0/4136136002 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18affd640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0b20a0 0x7fc18c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18b7fe640 1 -- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc18c0b2ad0 msgr2=0x7fc18c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18b7fe640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc18c0b2ad0 0x7fc18c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18b7fe640 1 -- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0b20a0 msgr2=0x7fc18c1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18b7fe640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0b20a0 0x7fc18c1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18b7fe640 1 -- 172.21.15.111:0/4136136002 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009400 con 0x7fc18c0d1780 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18affd640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0b20a0 0x7fc18c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc18b7fe640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c1f70c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc18400c7a0 tx=0x7fc18400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184003b70 con 0x7fc18c0d1780 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc184003cf0 con 0x7fc18c0d1780 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c151c00 con 0x7fc18c0d1780 2023-12-04T05:40:39.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184003ea0 con 0x7fc18c0d1780 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.134+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c152920 con 0x7fc18c0d1780 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.135+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c0693d0 con 0x7fc18c0d1780 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.135+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc18400aa90 con 0x7fc18c0d1780 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.136+0000 7fc1737fe640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc17806b2a0 0x7fc17806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.136+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc184089a00 con 0x7fc18c0d1780 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.136+0000 7fc18affd640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc17806b2a0 0x7fc17806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.136+0000 7fc18affd640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc17806b2a0 0x7fc17806d760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc17c036000 tx=0x7fc17c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.140 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.140+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc184054870 con 0x7fc18c0d1780 2023-12-04T05:40:39.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.308+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc18c0d40b0 con 0x7fc18c0d1780 2023-12-04T05:40:39.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.309+0000 7fc1737fe640 1 -- 172.21.15.111:0/4136136002 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fc184057e90 con 0x7fc18c0d1780 2023-12-04T05:40:39.310 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:39.310 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc17806b2a0 msgr2=0x7fc17806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc17806b2a0 0x7fc17806d760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc17c036000 tx=0x7fc17c008030 comp rx=0 tx=0).stop 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 msgr2=0x7fc18c1f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c1f70c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc18400c7a0 tx=0x7fc18400cc60 comp rx=0 tx=0).stop 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 shutdown_connections 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc17806b2a0 0x7fc17806d760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc18c0d1780 0x7fc18c1f70c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc18c0b2ad0 0x7fc18c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 --2- 172.21.15.111:0/4136136002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc18c0b20a0 0x7fc18c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 >> 172.21.15.111:0/4136136002 conn(0x7fc18c09c4e0 msgr2=0x7fc18c06cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 shutdown_connections 2023-12-04T05:40:39.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.311+0000 7fc19171d640 1 -- 172.21.15.111:0/4136136002 wait complete. 2023-12-04T05:40:39.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:39.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:39.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.425+0000 7f09e574c640 1 Processor -- start 2023-12-04T05:40:39.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.425+0000 7f09e574c640 1 -- start start 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09e574c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cfaf0 0x7f09e00cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09e574c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f09e00cbf50 0x7f09e015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09e574c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e0070190 con 0x7f09e015e770 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09e574c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e00cff50 con 0x7f09e00cbf50 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09e574c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00703e0 con 0x7f09e00cfaf0 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09de7fc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cfaf0 0x7f09e00cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09df7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f09e00cbf50 0x7f09e015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09deffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f09e015e770 0x7f09e009a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:36440/0 (socket says 172.21.15.111:36440) 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09df7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f09e00cbf50 0x7f09e015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34642/0 (socket says 172.21.15.111:34642) 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09deffd640 1 -- 172.21.15.111:0/4140937505 learned_addr learned my addr 172.21.15.111:0/4140937505 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09de7fc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cfaf0 0x7f09e00cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:55750/0 (socket says 172.21.15.111:55750) 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09df7fe640 1 -- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cfaf0 msgr2=0x7f09e00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09df7fe640 1 --2- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cfaf0 0x7f09e00cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09df7fe640 1 -- 172.21.15.111:0/4140937505 >> v1:172.21.15.192:6789/0 conn(0x7f09e015e770 legacy=0x7f09e009a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.426+0000 7f09df7fe640 1 -- 172.21.15.111:0/4140937505 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0099ba0 con 0x7f09e00cbf50 2023-12-04T05:40:39.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.427+0000 7f09df7fe640 1 --2- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e00cbf50 0x7f09e015ee50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097a0 tx=0x7f09d802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08683cc5c578778 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.427+0000 7f09ddffb640 1 -- 172.21.15.111:0/4140937505 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d802e070 con 0x7f09e00cbf50 2023-12-04T05:40:39.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.427+0000 7f09ddffb640 1 -- 172.21.15.111:0/4140937505 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d80315a0 con 0x7f09e00cbf50 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.427+0000 7f09ddffb640 1 -- 172.21.15.111:0/4140937505 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8031930 con 0x7f09e00cbf50 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 -- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e00cbf50 msgr2=0x7f09e015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 --2- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e00cbf50 0x7f09e015ee50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097a0 tx=0x7f09d802b840 comp rx=0 tx=0).stop 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 -- 172.21.15.111:0/4140937505 shutdown_connections 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 --2- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e00cbf50 0x7f09e015ee50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 --2- 172.21.15.111:0/4140937505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cfaf0 0x7f09e00cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 -- 172.21.15.111:0/4140937505 >> 172.21.15.111:0/4140937505 conn(0x7f09e00b2ad0 msgr2=0x7f09e00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 -- 172.21.15.111:0/4140937505 shutdown_connections 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.428+0000 7f09e574c640 1 -- 172.21.15.111:0/4140937505 wait complete. 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.429+0000 7f09e574c640 1 Processor -- start 2023-12-04T05:40:39.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.429+0000 7f09e574c640 1 -- start start 2023-12-04T05:40:39.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09e574c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cbf50 0x7f09e01586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09e574c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 0x7f09e00c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09e574c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e015e770 0x7f09e01f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09e574c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01f4660 con 0x7f09e00cfaf0 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09e574c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01f4940 con 0x7f09e015e770 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09e574c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01f4c20 con 0x7f09e00cbf50 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 0x7f09e00c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 0x7f09e00c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59976/0 (socket says 172.21.15.111:59976) 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 -- 172.21.15.111:0/667899361 learned_addr learned my addr 172.21.15.111:0/667899361 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09df7fe640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e015e770 0x7f09e01f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09deffd640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cbf50 0x7f09e01586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 -- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cbf50 msgr2=0x7f09e01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cbf50 0x7f09e01586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 -- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e015e770 msgr2=0x7f09e01f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e015e770 0x7f09e01f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09de7fc640 1 -- 172.21.15.111:0/667899361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d8009400 con 0x7f09e00cfaf0 2023-12-04T05:40:39.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.430+0000 7f09df7fe640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e015e770 0x7f09e01f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.431+0000 7f09de7fc640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 0x7f09e00c9990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e9b0 tx=0x7f09d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.431+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4009670 con 0x7f09e00cfaf0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.431+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d400c9a0 con 0x7f09e00cfaf0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.431+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400cd50 con 0x7f09e00cfaf0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.431+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e01f4f00 con 0x7f09e00cfaf0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.431+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e00c9ed0 con 0x7f09e00cfaf0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.432+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e00671a0 con 0x7f09e00cfaf0 2023-12-04T05:40:39.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.432+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f09d4013470 con 0x7f09e00cfaf0 2023-12-04T05:40:39.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.432+0000 7f09c77fe640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f09cc06b2a0 0x7f09cc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.433+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09d4089e20 con 0x7f09e00cfaf0 2023-12-04T05:40:39.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.433+0000 7f09deffd640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f09cc06b2a0 0x7f09cc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.433+0000 7f09deffd640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f09cc06b2a0 0x7f09cc06d760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f09d0005c00 tx=0x7f09d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.437+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09d4054c90 con 0x7f09e00cfaf0 2023-12-04T05:40:39.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.604+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09e0067460 con 0x7f09e00cfaf0 2023-12-04T05:40:39.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.605+0000 7f09c77fe640 1 -- 172.21.15.111:0/667899361 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09d40170a0 con 0x7f09e00cfaf0 2023-12-04T05:40:39.605 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:39.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f09cc06b2a0 msgr2=0x7f09cc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f09cc06b2a0 0x7f09cc06d760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f09d0005c00 tx=0x7f09d0009040 comp rx=0 tx=0).stop 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 msgr2=0x7f09e00c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 0x7f09e00c9990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e9b0 tx=0x7f09d400ee70 comp rx=0 tx=0).stop 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 shutdown_connections 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f09cc06b2a0 0x7f09cc06d760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09e015e770 0x7f09e01f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09e00cfaf0 0x7f09e00c9990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 --2- 172.21.15.111:0/667899361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09e00cbf50 0x7f09e01586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 >> 172.21.15.111:0/667899361 conn(0x7f09e00b2ad0 msgr2=0x7f09e00b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 shutdown_connections 2023-12-04T05:40:39.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.607+0000 7f09e574c640 1 -- 172.21.15.111:0/667899361 wait complete. 2023-12-04T05:40:39.618 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:39.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.719+0000 7fba3463d640 1 Processor -- start 2023-12-04T05:40:39.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.719+0000 7fba3463d640 1 -- start start 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.720+0000 7fba3463d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d860 0x7fba2c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.720+0000 7fba3463d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba2c0d0010 0x7fba2c160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.720+0000 7fba3463d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c15ddc0 con 0x7fba2c15d480 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.720+0000 7fba3463d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0d04a0 con 0x7fba2c0d0010 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.720+0000 7fba3463d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c15e0a0 con 0x7fba2c15d860 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba32bda640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba2c0d0010 0x7fba2c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba323d9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fba2c15d480 0x7fba2c0d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52182/0 (socket says 172.21.15.111:52182) 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba323d9640 1 -- 172.21.15.111:0/3673363396 learned_addr learned my addr 172.21.15.111:0/3673363396 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:39.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba31bd8640 1 --2- 172.21.15.111:0/3673363396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d860 0x7fba2c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba32bda640 1 -- 172.21.15.111:0/3673363396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d860 msgr2=0x7fba2c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba32bda640 1 --2- 172.21.15.111:0/3673363396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d860 0x7fba2c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba32bda640 1 -- 172.21.15.111:0/3673363396 >> v1:172.21.15.111:6789/0 conn(0x7fba2c15d480 legacy=0x7fba2c0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba313d7640 1 -- 172.21.15.111:0/3673363396 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919935902 0 0) 0x7fba2c15ddc0 con 0x7fba2c15d480 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.721+0000 7fba32bda640 1 -- 172.21.15.111:0/3673363396 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c15e380 con 0x7fba2c0d0010 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba32bda640 1 --2- 172.21.15.111:0/3673363396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c160270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fba24009d90 tx=0x7fba2402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de5af126cfcdc3b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba313d7640 1 -- 172.21.15.111:0/3673363396 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2402cc50 con 0x7fba2c0d0010 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba313d7640 1 -- 172.21.15.111:0/3673363396 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2402cdb0 con 0x7fba2c0d0010 2023-12-04T05:40:39.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba313d7640 1 -- 172.21.15.111:0/3673363396 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24034660 con 0x7fba2c0d0010 2023-12-04T05:40:39.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 -- 172.21.15.111:0/3673363396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 msgr2=0x7fba2c160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 --2- 172.21.15.111:0/3673363396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c160270 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fba24009d90 tx=0x7fba2402bea0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 -- 172.21.15.111:0/3673363396 shutdown_connections 2023-12-04T05:40:39.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 --2- 172.21.15.111:0/3673363396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c160270 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 --2- 172.21.15.111:0/3673363396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d860 0x7fba2c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 -- 172.21.15.111:0/3673363396 >> 172.21.15.111:0/3673363396 conn(0x7fba2c09c540 msgr2=0x7fba2c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:39.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 -- 172.21.15.111:0/3673363396 shutdown_connections 2023-12-04T05:40:39.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.722+0000 7fba3463d640 1 -- 172.21.15.111:0/3673363396 wait complete. 2023-12-04T05:40:39.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 Processor -- start 2023-12-04T05:40:39.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 -- start start 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c14c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d480 0x7fba2c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba2c15d860 0x7fba2c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c150b80 con 0x7fba2c0d0010 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0a0f90 con 0x7fba2c15d860 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba3463d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c14ff70 con 0x7fba2c15d480 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba323d9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c14c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba323d9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c14c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59984/0 (socket says 172.21.15.111:59984) 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.724+0000 7fba323d9640 1 -- 172.21.15.111:0/2513053669 learned_addr learned my addr 172.21.15.111:0/2513053669 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba323d9640 1 -- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d480 msgr2=0x7fba2c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba31bd8640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d480 0x7fba2c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba32bda640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba2c15d860 0x7fba2c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba323d9640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d480 0x7fba2c14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba323d9640 1 -- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba2c15d860 msgr2=0x7fba2c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba323d9640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba2c15d860 0x7fba2c151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba323d9640 1 -- 172.21.15.111:0/2513053669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009a70 con 0x7fba2c0d0010 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba31bd8640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d480 0x7fba2c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba32bda640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba2c15d860 0x7fba2c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba323d9640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c14c070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c4d0 tx=0x7fba2800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba280074e0 con 0x7fba2c0d0010 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba28007660 con 0x7fba2c0d0010 2023-12-04T05:40:39.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c150250 con 0x7fba2c0d0010 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28004a20 con 0x7fba2c0d0010 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.725+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c150810 con 0x7fba2c0d0010 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.726+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c067980 con 0x7fba2c0d0010 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.726+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fba28004b80 con 0x7fba2c0d0010 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.727+0000 7fba22ffd640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba1006b2d0 0x7fba1006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.727+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba28089020 con 0x7fba2c0d0010 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.727+0000 7fba31bd8640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba1006b2d0 0x7fba1006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:39.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.727+0000 7fba31bd8640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba1006b2d0 0x7fba1006d790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fba1c006fd0 tx=0x7fba1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:39.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.731+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba28053f20 con 0x7fba2c0d0010 2023-12-04T05:40:39.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.896+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba2c069360 con 0x7fba2c0d0010 2023-12-04T05:40:39.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.896+0000 7fba22ffd640 1 -- 172.21.15.111:0/2513053669 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba28018020 con 0x7fba2c0d0010 2023-12-04T05:40:39.898 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:39.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba1006b2d0 msgr2=0x7fba1006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba1006b2d0 0x7fba1006d790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fba1c006fd0 tx=0x7fba1c008040 comp rx=0 tx=0).stop 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 msgr2=0x7fba2c14c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c14c070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c4d0 tx=0x7fba2800c990 comp rx=0 tx=0).stop 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 shutdown_connections 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fba1006b2d0 0x7fba1006d790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba2c15d860 0x7fba2c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba2c15d480 0x7fba2c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 --2- 172.21.15.111:0/2513053669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba2c0d0010 0x7fba2c14c070 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 >> 172.21.15.111:0/2513053669 conn(0x7fba2c09c540 msgr2=0x7fba2c0b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 shutdown_connections 2023-12-04T05:40:39.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:39.898+0000 7fba3463d640 1 -- 172.21.15.111:0/2513053669 wait complete. 2023-12-04T05:40:39.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.011+0000 7f5c74a75640 1 Processor -- start 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.011+0000 7f5c74a75640 1 -- start start 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c74a75640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5c70070110 0x7f5c7006f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c74a75640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c700cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c74a75640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c700cfd60 con 0x7f5c70099a00 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c74a75640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7009a5d0 con 0x7f5c70070110 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c74a75640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7009a8b0 con 0x7f5c700cc3a0 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6e59c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5c70099a00 0x7f5c70099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52188/0 (socket says 172.21.15.111:52188) 2023-12-04T05:40:40.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6e59c640 1 -- 172.21.15.111:0/2377546421 learned_addr learned my addr 172.21.15.111:0/2377546421 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709325878 0 0) 0x7f5c700cfd60 con 0x7f5c70099a00 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003520 con 0x7f5c70099a00 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/2377546421 >> v2:172.21.15.192:3300/0 conn(0x7f5c70070110 0x7f5c7006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6ed9d640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c700cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 784381804 0 0) 0x7f5c58003520 con 0x7f5c70099a00 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 msgr2=0x7f5c700cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c700cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 >> v2:172.21.15.192:3300/0 conn(0x7f5c70070110 msgr2=0x7f5c7006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 --2- 172.21.15.111:0/2377546421 >> v2:172.21.15.192:3300/0 conn(0x7f5c70070110 0x7f5c7006f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700cf560 con 0x7f5c70099a00 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f5c68001670 con 0x7f5c70099a00 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 >> v1:172.21.15.111:6789/0 conn(0x7f5c70099a00 legacy=0x7f5c70099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c58003970 0x7f5c58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c580047c0 0x7f5c58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.012+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7009a5d0 con 0x7f5c580047c0 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7009a8b0 con 0x7f5c58003970 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5800b980 con 0x7f5c700cc3a0 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c580047c0 0x7f5c58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6e59c640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c58003970 0x7f5c58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6ed9d640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 msgr2=0x7f5c58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c58005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c58003970 msgr2=0x7f5c58003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c58003970 0x7f5c58003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/2377546421 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c58003520 con 0x7f5c580047c0 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.013+0000 7f5c6ed9d640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700cc3a0 0x7f5c58005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:40.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c580047c0 0x7f5c58004c40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009fc0 tx=0x7f5c580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f61afe94f421a4f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6402cc60 con 0x7f5c580047c0 2023-12-04T05:40:40.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400f040 con 0x7f5c580047c0 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c6d59a640 1 -- 172.21.15.111:0/2377546421 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64034440 con 0x7f5c580047c0 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c74a75640 1 -- 172.21.15.111:0/2377546421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c580047c0 msgr2=0x7f5c58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c74a75640 1 --2- 172.21.15.111:0/2377546421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c580047c0 0x7f5c58004c40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009fc0 tx=0x7f5c580057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c6ed9d640 1 -- 172.21.15.111:0/2377546421 reap_dead start 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c74a75640 1 -- 172.21.15.111:0/2377546421 shutdown_connections 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c74a75640 1 -- 172.21.15.111:0/2377546421 >> 172.21.15.111:0/2377546421 conn(0x7f5c700b2a70 msgr2=0x7f5c700b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c74a75640 1 -- 172.21.15.111:0/2377546421 shutdown_connections 2023-12-04T05:40:40.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.014+0000 7f5c74a75640 1 -- 172.21.15.111:0/2377546421 wait complete. 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 Processor -- start 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 -- start start 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c580047c0 0x7f5c701535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 0x7f5c700b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b4be0 0x7f5c700af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70153470 con 0x7f5c70152ff0 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700b5440 con 0x7f5c580047c0 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.016+0000 7f5c74a75640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700b5160 con 0x7f5c700b4be0 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 0x7f5c700b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 0x7f5c700b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60002/0 (socket says 172.21.15.111:60002) 2023-12-04T05:40:40.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/3596864650 learned_addr learned my addr 172.21.15.111:0/3596864650 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6e59c640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c580047c0 0x7f5c701535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6ed9d640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b4be0 0x7f5c700af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b4be0 msgr2=0x7f5c700af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b4be0 0x7f5c700af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c580047c0 msgr2=0x7f5c701535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c580047c0 0x7f5c701535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 -- 172.21.15.111:0/3596864650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700c96f0 con 0x7f5c70152ff0 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6e59c640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c580047c0 0x7f5c701535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c6dd9b640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 0x7f5c700b46a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e230 tx=0x7f5c64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64034c50 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c64034db0 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c700c99d0 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64003bd0 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.017+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c701f51f0 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.018+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c700c9530 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.019+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5c64040450 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.019+0000 7f5c5effd640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5006b2d0 0x7f5c5006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.019+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c640ac310 con 0x7f5c70152ff0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.019+0000 7f5c6e59c640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5006b2d0 0x7f5c5006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.019+0000 7f5c6e59c640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5006b2d0 0x7f5c5006d790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007d40 tx=0x7f5c68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.023+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c64077210 con 0x7f5c70152ff0 2023-12-04T05:40:40.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.187+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c70069360 con 0x7f5c70152ff0 2023-12-04T05:40:40.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.187+0000 7f5c5effd640 1 -- 172.21.15.111:0/3596864650 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c6407a830 con 0x7f5c70152ff0 2023-12-04T05:40:40.189 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5006b2d0 msgr2=0x7f5c5006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5006b2d0 0x7f5c5006d790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007d40 tx=0x7f5c68009040 comp rx=0 tx=0).stop 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 msgr2=0x7f5c700b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 0x7f5c700b46a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e230 tx=0x7f5c64009750 comp rx=0 tx=0).stop 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 shutdown_connections 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5c5006b2d0 0x7f5c5006d790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b4be0 0x7f5c700af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c70152ff0 0x7f5c700b46a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 --2- 172.21.15.111:0/3596864650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c580047c0 0x7f5c701535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 >> 172.21.15.111:0/3596864650 conn(0x7f5c700b2a70 msgr2=0x7f5c7015cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 shutdown_connections 2023-12-04T05:40:40.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.189+0000 7f5c74a75640 1 -- 172.21.15.111:0/3596864650 wait complete. 2023-12-04T05:40:40.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:40:40.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.301+0000 7f8499261640 1 Processor -- start 2023-12-04T05:40:40.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.301+0000 7f8499261640 1 -- start start 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f8499261640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f849415aba0 0x7f8494164e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f8499261640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84940aea20 con 0x7f849413cee0 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f8499261640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849413d2c0 con 0x7f849406c960 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f8499261640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849415b3a0 con 0x7f849415aba0 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f84937fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f849415aba0 0x7f8494164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f8492ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f849413cee0 0x7f849406c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:52204/0 (socket says 172.21.15.111:52204) 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f8492ffd640 1 -- 172.21.15.111:0/342591493 learned_addr learned my addr 172.21.15.111:0/342591493 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f84937fe640 1 -- 172.21.15.111:0/342591493 >> v1:172.21.15.192:6789/0 conn(0x7f849406c960 legacy=0x7f849415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f84937fe640 1 -- 172.21.15.111:0/342591493 >> v1:172.21.15.111:6789/0 conn(0x7f849413cee0 legacy=0x7f849406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f84937fe640 1 -- 172.21.15.111:0/342591493 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849415b0c0 con 0x7f849415aba0 2023-12-04T05:40:40.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.302+0000 7f84927fc640 1 -- 172.21.15.111:0/342591493 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009481153 0 0) 0x7f84940aea20 con 0x7f849413cee0 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.303+0000 7f84937fe640 1 --2- 172.21.15.111:0/342591493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849415aba0 0x7f8494164e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f848c009d90 tx=0x7f848c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15868734101308c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.303+0000 7f84927fc640 1 -- 172.21.15.111:0/342591493 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c02cc60 con 0x7f849415aba0 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.303+0000 7f84927fc640 1 -- 172.21.15.111:0/342591493 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c02cdc0 con 0x7f849415aba0 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.303+0000 7f84927fc640 1 -- 172.21.15.111:0/342591493 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c034660 con 0x7f849415aba0 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.303+0000 7f8499261640 1 -- 172.21.15.111:0/342591493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849415aba0 msgr2=0x7f8494164e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.303+0000 7f8499261640 1 --2- 172.21.15.111:0/342591493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849415aba0 0x7f8494164e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f848c009d90 tx=0x7f848c02beb0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.304+0000 7f8499261640 1 -- 172.21.15.111:0/342591493 shutdown_connections 2023-12-04T05:40:40.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.304+0000 7f8499261640 1 --2- 172.21.15.111:0/342591493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849415aba0 0x7f8494164e10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.304+0000 7f8499261640 1 -- 172.21.15.111:0/342591493 >> 172.21.15.111:0/342591493 conn(0x7f849406a4c0 msgr2=0x7f849406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.304+0000 7f8499261640 1 -- 172.21.15.111:0/342591493 shutdown_connections 2023-12-04T05:40:40.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.304+0000 7f8499261640 1 -- 172.21.15.111:0/342591493 wait complete. 2023-12-04T05:40:40.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.305+0000 7f8499261640 1 Processor -- start 2023-12-04T05:40:40.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.305+0000 7f8499261640 1 -- start start 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f849406c960 0x7f84941f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 0x7f84941f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f849415aba0 0x7f84940703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8494158d40 con 0x7f849413cee0 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941f3490 con 0x7f849415aba0 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84941580c0 con 0x7f849406c960 2023-12-04T05:40:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 0x7f84941f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 0x7f84941f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60022/0 (socket says 172.21.15.111:60022) 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 -- 172.21.15.111:0/2491736483 learned_addr learned my addr 172.21.15.111:0/2491736483 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f84937fe640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f849415aba0 0x7f84940703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8492ffd640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f849406c960 0x7f84941f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 -- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f849406c960 msgr2=0x7f84941f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f849406c960 0x7f84941f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 -- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f849415aba0 msgr2=0x7f84940703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f849415aba0 0x7f84940703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 -- 172.21.15.111:0/2491736483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009a70 con 0x7f849413cee0 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848a7fc640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 0x7f84941f3e50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c450 tx=0x7f847c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c0052d0 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c005430 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84941583a0 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c012870 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.306+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8494158960 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.307+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849415aa10 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.308+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f847c0129d0 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.308+0000 7f848bfff640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f847406b2a0 0x7f847406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.308+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f847c089360 con 0x7f849413cee0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.308+0000 7f8492ffd640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f847406b2a0 0x7f847406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.309+0000 7f8492ffd640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f847406b2a0 0x7f847406d760 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8484007f70 tx=0x7f8484007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.312+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f847c054310 con 0x7f849413cee0 2023-12-04T05:40:40.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.491+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8494158e80 con 0x7f849413cee0 2023-12-04T05:40:40.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.491+0000 7f848bfff640 1 -- 172.21.15.111:0/2491736483 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8494158e80 con 0x7f849413cee0 2023-12-04T05:40:40.493 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:40:40.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f847406b2a0 msgr2=0x7f847406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f847406b2a0 0x7f847406d760 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8484007f70 tx=0x7f8484007f00 comp rx=0 tx=0).stop 2023-12-04T05:40:40.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 msgr2=0x7f84941f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 0x7f84941f3e50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c450 tx=0x7f847c00c910 comp rx=0 tx=0).stop 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 shutdown_connections 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f847406b2a0 0x7f847406d760 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f849415aba0 0x7f84940703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f849413cee0 0x7f84941f3e50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 --2- 172.21.15.111:0/2491736483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f849406c960 0x7f84941f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 >> 172.21.15.111:0/2491736483 conn(0x7f849406a4c0 msgr2=0x7f84940cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 shutdown_connections 2023-12-04T05:40:40.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.494+0000 7f8499261640 1 -- 172.21.15.111:0/2491736483 wait complete. 2023-12-04T05:40:40.506 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:40.506 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:40.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:40.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.659+0000 7fce6070f640 1 Processor -- start 2023-12-04T05:40:40.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.659+0000 7fce6070f640 1 -- start start 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce6070f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce58144430 0x7fce5815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce6070f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fce5806c250 0x7fce5806c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce6070f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce58070bd0 con 0x7fce580703f0 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce6070f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce58144810 con 0x7fce5806c250 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce6070f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5806cc20 con 0x7fce58144430 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce5dcaa640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fce580703f0 0x7fce58071c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53834/0 (socket says 172.21.15.111:53834) 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce5dcaa640 1 -- 172.21.15.111:0/2102025991 learned_addr learned my addr 172.21.15.111:0/2102025991 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce5e4ab640 1 --2- 172.21.15.111:0/2102025991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce58144430 0x7fce5815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.660+0000 7fce5ecac640 1 --2- 172.21.15.111:0/2102025991 >> v2:172.21.15.192:3300/0 conn(0x7fce5806c250 0x7fce5806c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5d4a9640 1 -- 172.21.15.111:0/2102025991 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615635039 0 0) 0x7fce58070bd0 con 0x7fce580703f0 2023-12-04T05:40:40.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5d4a9640 1 -- 172.21.15.111:0/2102025991 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce44003520 con 0x7fce580703f0 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5d4a9640 1 -- 172.21.15.111:0/2102025991 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1449882664 0 0) 0x7fce44003520 con 0x7fce580703f0 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5e4ab640 1 -- 172.21.15.111:0/2102025991 >> v2:172.21.15.192:3300/0 conn(0x7fce5806c250 msgr2=0x7fce5806c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5e4ab640 1 --2- 172.21.15.111:0/2102025991 >> v2:172.21.15.192:3300/0 conn(0x7fce5806c250 0x7fce5806c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5e4ab640 1 -- 172.21.15.111:0/2102025991 >> v1:172.21.15.111:6789/0 conn(0x7fce580703f0 legacy=0x7fce58071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5e4ab640 1 -- 172.21.15.111:0/2102025991 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58070eb0 con 0x7fce58144430 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.661+0000 7fce5ecac640 1 --2- 172.21.15.111:0/2102025991 >> v2:172.21.15.192:3300/0 conn(0x7fce5806c250 0x7fce5806c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce5e4ab640 1 --2- 172.21.15.111:0/2102025991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce58144430 0x7fce5815f7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce54009d90 tx=0x7fce5402bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec18a95d5dcf91d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce5d4a9640 1 -- 172.21.15.111:0/2102025991 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48001670 con 0x7fce58144430 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce5d4a9640 1 -- 172.21.15.111:0/2102025991 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5402c8a0 con 0x7fce58144430 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce5d4a9640 1 -- 172.21.15.111:0/2102025991 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5402cc10 con 0x7fce58144430 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 -- 172.21.15.111:0/2102025991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce58144430 msgr2=0x7fce5815f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 --2- 172.21.15.111:0/2102025991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce58144430 0x7fce5815f7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce54009d90 tx=0x7fce5402bd70 comp rx=0 tx=0).stop 2023-12-04T05:40:40.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 -- 172.21.15.111:0/2102025991 shutdown_connections 2023-12-04T05:40:40.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 --2- 172.21.15.111:0/2102025991 >> v2:172.21.15.192:3300/0 conn(0x7fce5806c250 0x7fce5806c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 --2- 172.21.15.111:0/2102025991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce58144430 0x7fce5815f7b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 -- 172.21.15.111:0/2102025991 >> 172.21.15.111:0/2102025991 conn(0x7fce5806a4c0 msgr2=0x7fce5806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 -- 172.21.15.111:0/2102025991 shutdown_connections 2023-12-04T05:40:40.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.662+0000 7fce6070f640 1 -- 172.21.15.111:0/2102025991 wait complete. 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.663+0000 7fce6070f640 1 Processor -- start 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 -- start start 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce5806c250 0x7fce58141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce580703f0 0x7fce580c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 0x7fce580c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c7f90 con 0x7fce58144430 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58142060 con 0x7fce580703f0 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce6070f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce58142290 con 0x7fce5806c250 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce5ecac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 0x7fce580c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce5ecac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 0x7fce580c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55992/0 (socket says 172.21.15.111:55992) 2023-12-04T05:40:40.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.664+0000 7fce5ecac640 1 -- 172.21.15.111:0/3561079980 learned_addr learned my addr 172.21.15.111:0/3561079980 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5e4ab640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce5806c250 0x7fce58141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5dcaa640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce580703f0 0x7fce580c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5ecac640 1 -- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce5806c250 msgr2=0x7fce58141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5ecac640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce5806c250 0x7fce58141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5ecac640 1 -- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce580703f0 msgr2=0x7fce580c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5ecac640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce580703f0 0x7fce580c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5ecac640 1 -- 172.21.15.111:0/3561079980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009400 con 0x7fce58144430 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce5ecac640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 0x7fce580c7a50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e880 tx=0x7fce5000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50009660 con 0x7fce58144430 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50004020 con 0x7fce58144430 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce54009a70 con 0x7fce58144430 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500043b0 con 0x7fce58144430 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.665+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce58142850 con 0x7fce58144430 2023-12-04T05:40:40.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.666+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce580693d0 con 0x7fce58144430 2023-12-04T05:40:40.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.666+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fce5000c940 con 0x7fce58144430 2023-12-04T05:40:40.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.667+0000 7fce4effd640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fce3c06b2d0 0x7fce3c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.667+0000 7fce5e4ab640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fce3c06b2d0 0x7fce3c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.667+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce50089910 con 0x7fce58144430 2023-12-04T05:40:40.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.667+0000 7fce5e4ab640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fce3c06b2d0 0x7fce3c06d790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce54036000 tx=0x7fce54030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.671+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce50054830 con 0x7fce58144430 2023-12-04T05:40:40.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.837+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce5815b230 con 0x7fce58144430 2023-12-04T05:40:40.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.838+0000 7fce4effd640 1 -- 172.21.15.111:0/3561079980 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fce50020090 con 0x7fce58144430 2023-12-04T05:40:40.839 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:40.839 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:40.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fce3c06b2d0 msgr2=0x7fce3c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fce3c06b2d0 0x7fce3c06d790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce54036000 tx=0x7fce54030000 comp rx=0 tx=0).stop 2023-12-04T05:40:40.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 msgr2=0x7fce580c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 0x7fce580c7a50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e880 tx=0x7fce5000ed40 comp rx=0 tx=0).stop 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 shutdown_connections 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fce3c06b2d0 0x7fce3c06d790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce58144430 0x7fce580c7a50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce580703f0 0x7fce580c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 --2- 172.21.15.111:0/3561079980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce5806c250 0x7fce58141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 >> 172.21.15.111:0/3561079980 conn(0x7fce5806a4c0 msgr2=0x7fce5815d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 shutdown_connections 2023-12-04T05:40:40.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.840+0000 7fce6070f640 1 -- 172.21.15.111:0/3561079980 wait complete. 2023-12-04T05:40:40.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:40.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:40.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.952+0000 7fe6b97ce640 1 Processor -- start 2023-12-04T05:40:40.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.952+0000 7fe6b97ce640 1 -- start start 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b97ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6b415d860 0x7fe6b40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b97ce640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b415ddc0 con 0x7fe6b415d480 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b97ce640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b415e0a0 con 0x7fe6b40d0010 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b97ce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40d04a0 con 0x7fe6b415d860 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b27fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6b415d860 0x7fe6b40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b2ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe6b415d480 0x7fe6b40d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53840/0 (socket says 172.21.15.111:53840) 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b27fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6b415d860 0x7fe6b40cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55994/0 (socket says 172.21.15.111:55994) 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b2ffd640 1 -- 172.21.15.111:0/2366836207 learned_addr learned my addr 172.21.15.111:0/2366836207 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b27fc640 1 -- 172.21.15.111:0/2366836207 >> v1:172.21.15.192:6789/0 conn(0x7fe6b40d0010 legacy=0x7fe6b4160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b27fc640 1 -- 172.21.15.111:0/2366836207 >> v1:172.21.15.111:6789/0 conn(0x7fe6b415d480 legacy=0x7fe6b40d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b1ffb640 1 -- 172.21.15.111:0/2366836207 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058842638 0 0) 0x7fe6b415ddc0 con 0x7fe6b415d480 2023-12-04T05:40:40.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.953+0000 7fe6b27fc640 1 -- 172.21.15.111:0/2366836207 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b415e380 con 0x7fe6b415d860 2023-12-04T05:40:40.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.954+0000 7fe6b27fc640 1 --2- 172.21.15.111:0/2366836207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d860 0x7fe6b40cfad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097d0 tx=0x7fe6a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529dd774a6e24084 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.954+0000 7fe6b1ffb640 1 -- 172.21.15.111:0/2366836207 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8031830 con 0x7fe6b415d860 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.954+0000 7fe6b1ffb640 1 -- 172.21.15.111:0/2366836207 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8031990 con 0x7fe6b415d860 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.954+0000 7fe6b1ffb640 1 -- 172.21.15.111:0/2366836207 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8031d20 con 0x7fe6b415d860 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 -- 172.21.15.111:0/2366836207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d860 msgr2=0x7fe6b40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/2366836207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d860 0x7fe6b40cfad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097d0 tx=0x7fe6a802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 -- 172.21.15.111:0/2366836207 shutdown_connections 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/2366836207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d860 0x7fe6b40cfad0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 -- 172.21.15.111:0/2366836207 >> 172.21.15.111:0/2366836207 conn(0x7fe6b409c540 msgr2=0x7fe6b40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 -- 172.21.15.111:0/2366836207 shutdown_connections 2023-12-04T05:40:40.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.955+0000 7fe6b97ce640 1 -- 172.21.15.111:0/2366836207 wait complete. 2023-12-04T05:40:40.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.956+0000 7fe6b97ce640 1 Processor -- start 2023-12-04T05:40:40.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.956+0000 7fe6b97ce640 1 -- start start 2023-12-04T05:40:40.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40d0010 0x7fe6b40b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 0x7fe6b41e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b415d860 0x7fe6b41e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41e7d20 con 0x7fe6b415d480 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40c88c0 con 0x7fe6b415d860 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40c8ba0 con 0x7fe6b40d0010 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 0x7fe6b41e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 0x7fe6b41e6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56004/0 (socket says 172.21.15.111:56004) 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 -- 172.21.15.111:0/562414620 learned_addr learned my addr 172.21.15.111:0/562414620 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b2ffd640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40d0010 0x7fe6b40b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b37fe640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b415d860 0x7fe6b41e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 -- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40d0010 msgr2=0x7fe6b40b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40d0010 0x7fe6b40b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 -- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b415d860 msgr2=0x7fe6b41e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b415d860 0x7fe6b41e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 -- 172.21.15.111:0/562414620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8009400 con 0x7fe6b415d480 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b27fc640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 0x7fe6b41e6db0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8034040 tx=0x7fe6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a803ba70 con 0x7fe6b415d480 2023-12-04T05:40:40.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a803bbf0 con 0x7fe6b415d480 2023-12-04T05:40:40.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b40b79c0 con 0x7fe6b415d480 2023-12-04T05:40:40.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.957+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80453f0 con 0x7fe6b415d480 2023-12-04T05:40:40.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.958+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b40b8700 con 0x7fe6b415d480 2023-12-04T05:40:40.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.959+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b4067980 con 0x7fe6b415d480 2023-12-04T05:40:40.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.959+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe6a804e450 con 0x7fe6b415d480 2023-12-04T05:40:40.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.959+0000 7fe69b7fe640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe6a006b2a0 0x7fe6a006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:40.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.959+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6a80abd10 con 0x7fe6b415d480 2023-12-04T05:40:40.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.959+0000 7fe6b2ffd640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe6a006b2a0 0x7fe6a006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:40.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.960+0000 7fe6b2ffd640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe6a006b2a0 0x7fe6a006d760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40012e0 tx=0x7fe6a4007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:40.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:40.964+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6a8076c10 con 0x7fe6b415d480 2023-12-04T05:40:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.130+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6b40b7ee0 con 0x7fe6b415d480 2023-12-04T05:40:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.131+0000 7fe69b7fe640 1 -- 172.21.15.111:0/562414620 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6b40b7ee0 con 0x7fe6b415d480 2023-12-04T05:40:41.133 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:41.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe6a006b2a0 msgr2=0x7fe6a006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe6a006b2a0 0x7fe6a006d760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40012e0 tx=0x7fe6a4007ef0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 msgr2=0x7fe6b41e6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 0x7fe6b41e6db0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8034040 tx=0x7fe6a80046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 shutdown_connections 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe6a006b2a0 0x7fe6a006d760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b415d860 0x7fe6b41e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6b415d480 0x7fe6b41e6db0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 --2- 172.21.15.111:0/562414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40d0010 0x7fe6b40b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 >> 172.21.15.111:0/562414620 conn(0x7fe6b409c540 msgr2=0x7fe6b40d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 shutdown_connections 2023-12-04T05:40:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.133+0000 7fe6b97ce640 1 -- 172.21.15.111:0/562414620 wait complete. 2023-12-04T05:40:41.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:41.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.247+0000 7f1195b4c640 1 Processor -- start 2023-12-04T05:40:41.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.247+0000 7f1195b4c640 1 -- start start 2023-12-04T05:40:41.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f1195b4c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1190099a00 0x7f119006f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f1195b4c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190070110 0x7f1190099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f1195b4c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119009a320 con 0x7f119006e2a0 2023-12-04T05:40:41.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f1195b4c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11900cfd60 con 0x7f1190099a00 2023-12-04T05:40:41.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f1195b4c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119009a5d0 con 0x7f1190070110 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1190099a00 0x7f119006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1190099a00 0x7f119006f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56012/0 (socket says 172.21.15.111:56012) 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 -- 172.21.15.111:0/3785319346 learned_addr learned my addr 172.21.15.111:0/3785319346 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118effd640 1 --2- 172.21.15.111:0/3785319346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190070110 0x7f1190099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 -- 172.21.15.111:0/3785319346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190070110 msgr2=0x7f1190099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 --2- 172.21.15.111:0/3785319346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190070110 0x7f1190099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 -- 172.21.15.111:0/3785319346 >> v1:172.21.15.192:6789/0 conn(0x7f119006e2a0 legacy=0x7f119006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.248+0000 7f118f7fe640 1 -- 172.21.15.111:0/3785319346 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119009a8b0 con 0x7f1190099a00 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f118f7fe640 1 --2- 172.21.15.111:0/3785319346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190099a00 0x7f119006f600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1184009870 tx=0x7f118402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3ee2fda457af20 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f118e7fc640 1 -- 172.21.15.111:0/3785319346 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184030840 con 0x7f1190099a00 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f118e7fc640 1 -- 172.21.15.111:0/3785319346 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11840309a0 con 0x7f1190099a00 2023-12-04T05:40:41.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f118e7fc640 1 -- 172.21.15.111:0/3785319346 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184030d30 con 0x7f1190099a00 2023-12-04T05:40:41.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 -- 172.21.15.111:0/3785319346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190099a00 msgr2=0x7f119006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 --2- 172.21.15.111:0/3785319346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190099a00 0x7f119006f600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1184009870 tx=0x7f118402b980 comp rx=0 tx=0).stop 2023-12-04T05:40:41.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 -- 172.21.15.111:0/3785319346 shutdown_connections 2023-12-04T05:40:41.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 --2- 172.21.15.111:0/3785319346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190070110 0x7f1190099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 --2- 172.21.15.111:0/3785319346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190099a00 0x7f119006f600 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 -- 172.21.15.111:0/3785319346 >> 172.21.15.111:0/3785319346 conn(0x7f11900b2a70 msgr2=0x7f11900b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 -- 172.21.15.111:0/3785319346 shutdown_connections 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.249+0000 7f1195b4c640 1 -- 172.21.15.111:0/3785319346 wait complete. 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 Processor -- start 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 -- start start 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119006e2a0 0x7f11900c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 0x7f11900c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190099a00 0x7f119015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11900b5510 con 0x7f1190070110 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119006d5c0 con 0x7f119006e2a0 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.251+0000 7f1195b4c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11900b4b70 con 0x7f1190099a00 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 0x7f11900c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 0x7f11900c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56014/0 (socket says 172.21.15.111:56014) 2023-12-04T05:40:41.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 -- 172.21.15.111:0/1672044723 learned_addr learned my addr 172.21.15.111:0/1672044723 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118f7fe640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119006e2a0 0x7f11900c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118ffff640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190099a00 0x7f119015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 -- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190099a00 msgr2=0x7f119015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190099a00 0x7f119015ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 -- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119006e2a0 msgr2=0x7f11900c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119006e2a0 0x7f11900c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 -- 172.21.15.111:0/1672044723 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840094d0 con 0x7f1190070110 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118f7fe640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119006e2a0 0x7f11900c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f118effd640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 0x7f11900c8ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9b0 tx=0x7f118000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180009670 con 0x7f1190070110 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000c9a0 con 0x7f1190070110 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000cd50 con 0x7f1190070110 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11900b4e50 con 0x7f1190070110 2023-12-04T05:40:41.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.252+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11900c4f50 con 0x7f1190070110 2023-12-04T05:40:41.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.253+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11900b5780 con 0x7f1190070110 2023-12-04T05:40:41.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.253+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1180013470 con 0x7f1190070110 2023-12-04T05:40:41.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.254+0000 7f1177fff640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f117c06b2a0 0x7f117c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.254+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1180089d50 con 0x7f1190070110 2023-12-04T05:40:41.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.254+0000 7f118f7fe640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f117c06b2a0 0x7f117c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.254+0000 7f118f7fe640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f117c06b2a0 0x7f117c06d760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1184036000 tx=0x7f118403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.258+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f118008d030 con 0x7f1190070110 2023-12-04T05:40:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.425+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1190069360 con 0x7f1190070110 2023-12-04T05:40:41.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.425+0000 7f1177fff640 1 -- 172.21.15.111:0/1672044723 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1180054c50 con 0x7f1190070110 2023-12-04T05:40:41.427 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f117c06b2a0 msgr2=0x7f117c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f117c06b2a0 0x7f117c06d760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1184036000 tx=0x7f118403b040 comp rx=0 tx=0).stop 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 msgr2=0x7f11900c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 0x7f11900c8ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9b0 tx=0x7f118000ee70 comp rx=0 tx=0).stop 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 shutdown_connections 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f117c06b2a0 0x7f117c06d760 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1190099a00 0x7f119015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1190070110 0x7f11900c8ea0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 --2- 172.21.15.111:0/1672044723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119006e2a0 0x7f11900c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 >> 172.21.15.111:0/1672044723 conn(0x7f11900b2a70 msgr2=0x7f119015be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:41.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 shutdown_connections 2023-12-04T05:40:41.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.427+0000 7f1195b4c640 1 -- 172.21.15.111:0/1672044723 wait complete. 2023-12-04T05:40:41.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:41.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.541+0000 7f573ae07640 1 Processor -- start 2023-12-04T05:40:41.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.541+0000 7f573ae07640 1 -- start start 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f573ae07640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f573413cee0 0x7f573413d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f573ae07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f573413fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f573ae07640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573413d2c0 con 0x7f573413f640 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f573ae07640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57341400e0 con 0x7f573413cee0 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f573ae07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57341403c0 con 0x7f5734072430 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5738ba3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f573413cee0 0x7f573413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5733fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f573413f640 0x7f5734071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53844/0 (socket says 172.21.15.111:53844) 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5733fff640 1 -- 172.21.15.111:0/3521775707 learned_addr learned my addr 172.21.15.111:0/3521775707 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:41.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5738ba3640 1 -- 172.21.15.111:0/3521775707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 msgr2=0x7f573413fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:41.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5738ba3640 1 --2- 172.21.15.111:0/3521775707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f573413fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5738ba3640 1 -- 172.21.15.111:0/3521775707 >> v1:172.21.15.111:6789/0 conn(0x7f573413f640 legacy=0x7f5734071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f57337fe640 1 -- 172.21.15.111:0/3521775707 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264499485 0 0) 0x7f573413d2c0 con 0x7f573413f640 2023-12-04T05:40:41.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.542+0000 7f5738ba3640 1 -- 172.21.15.111:0/3521775707 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340cdb50 con 0x7f573413cee0 2023-12-04T05:40:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.543+0000 7f5738ba3640 1 --2- 172.21.15.111:0/3521775707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413cee0 0x7f573413d600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57240097a0 tx=0x7f572402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ba8dcd4a809ffc server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.543+0000 7f57337fe640 1 -- 172.21.15.111:0/3521775707 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572402f050 con 0x7f573413cee0 2023-12-04T05:40:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.543+0000 7f57337fe640 1 -- 172.21.15.111:0/3521775707 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724031700 con 0x7f573413cee0 2023-12-04T05:40:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.543+0000 7f57337fe640 1 -- 172.21.15.111:0/3521775707 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724031a90 con 0x7f573413cee0 2023-12-04T05:40:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 -- 172.21.15.111:0/3521775707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413cee0 msgr2=0x7f573413d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 --2- 172.21.15.111:0/3521775707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413cee0 0x7f573413d600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57240097a0 tx=0x7f572402b7a0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 -- 172.21.15.111:0/3521775707 shutdown_connections 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 --2- 172.21.15.111:0/3521775707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f573413fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 --2- 172.21.15.111:0/3521775707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413cee0 0x7f573413d600 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 -- 172.21.15.111:0/3521775707 >> 172.21.15.111:0/3521775707 conn(0x7f57340b0cf0 msgr2=0x7f57340b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 -- 172.21.15.111:0/3521775707 shutdown_connections 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.544+0000 7f573ae07640 1 -- 172.21.15.111:0/3521775707 wait complete. 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.545+0000 7f573ae07640 1 Processor -- start 2023-12-04T05:40:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.545+0000 7f573ae07640 1 -- start start 2023-12-04T05:40:41.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f573ae07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f57341f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f573ae07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f573413cee0 0x7f573413b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f573ae07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 0x7f57341f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f573ae07640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57341f1650 con 0x7f573413f640 2023-12-04T05:40:41.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f573ae07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57341f1930 con 0x7f573413cee0 2023-12-04T05:40:41.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f573ae07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57341f4150 con 0x7f5734072430 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 0x7f57341f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 0x7f57341f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56030/0 (socket says 172.21.15.111:56030) 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 -- 172.21.15.111:0/2690247269 learned_addr learned my addr 172.21.15.111:0/2690247269 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f5733fff640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f573413cee0 0x7f573413b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f5738ba3640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f57341f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 -- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 msgr2=0x7f57341f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f57341f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 -- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f573413cee0 msgr2=0x7f573413b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f573413cee0 0x7f573413b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 -- 172.21.15.111:0/2690247269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5724009400 con 0x7f573413f640 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f5738ba3640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f57341f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57393a4640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 0x7f57341f0b60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f572c00b970 tx=0x7f572c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00cb40 con 0x7f573413f640 2023-12-04T05:40:41.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.546+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00ccc0 con 0x7f573413f640 2023-12-04T05:40:41.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.547+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57341f4430 con 0x7f573413f640 2023-12-04T05:40:41.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.547+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c014680 con 0x7f573413f640 2023-12-04T05:40:41.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.547+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5734071590 con 0x7f573413f640 2023-12-04T05:40:41.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.547+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734069360 con 0x7f573413f640 2023-12-04T05:40:41.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.548+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f572c01d3b0 con 0x7f573413f640 2023-12-04T05:40:41.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.548+0000 7f57317fa640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f572006b2a0 0x7f572006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.548+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f572c089b80 con 0x7f573413f640 2023-12-04T05:40:41.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.548+0000 7f5738ba3640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f572006b2a0 0x7f572006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.549+0000 7f5738ba3640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f572006b2a0 0x7f572006d760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5724035000 tx=0x7f5724008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.553+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f572c0549f0 con 0x7f573413f640 2023-12-04T05:40:41.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.719+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5734070380 con 0x7f573413f640 2023-12-04T05:40:41.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.720+0000 7f57317fa640 1 -- 172.21.15.111:0/2690247269 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f572c019020 con 0x7f573413f640 2023-12-04T05:40:41.721 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:41.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f572006b2a0 msgr2=0x7f572006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f572006b2a0 0x7f572006d760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5724035000 tx=0x7f5724008030 comp rx=0 tx=0).stop 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 msgr2=0x7f57341f0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 0x7f57341f0b60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f572c00b970 tx=0x7f572c00be30 comp rx=0 tx=0).stop 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 shutdown_connections 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f572006b2a0 0x7f572006d760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f573413f640 0x7f57341f0b60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f573413cee0 0x7f573413b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 --2- 172.21.15.111:0/2690247269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5734072430 0x7f57341f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 >> 172.21.15.111:0/2690247269 conn(0x7f57340b0cf0 msgr2=0x7f57340aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 shutdown_connections 2023-12-04T05:40:41.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.722+0000 7f573ae07640 1 -- 172.21.15.111:0/2690247269 wait complete. 2023-12-04T05:40:41.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:40:41.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.833+0000 7fb1938cf640 1 Processor -- start 2023-12-04T05:40:41.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.833+0000 7fb1938cf640 1 -- start start 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb1938cf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb18c1544c0 0x7fb18c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb1938cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb1938cf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c06df90 con 0x7fb18c06ba10 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb1938cf640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c06bec0 con 0x7fb18c1544c0 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb1938cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0ad5d0 con 0x7fb18c06d250 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb18c1544c0 0x7fb18c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb18c1544c0 0x7fb18c06cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56042/0 (socket says 172.21.15.111:56042) 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 -- 172.21.15.111:0/2792837310 learned_addr learned my addr 172.21.15.111:0/2792837310 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb190e6a640 1 --2- 172.21.15.111:0/2792837310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 -- 172.21.15.111:0/2792837310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 msgr2=0x7fb18c06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 --2- 172.21.15.111:0/2792837310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c06b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 -- 172.21.15.111:0/2792837310 >> v1:172.21.15.192:6789/0 conn(0x7fb18c06ba10 legacy=0x7fb18c160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.834+0000 7fb19166b640 1 -- 172.21.15.111:0/2792837310 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c1548a0 con 0x7fb18c1544c0 2023-12-04T05:40:41.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.835+0000 7fb19166b640 1 --2- 172.21.15.111:0/2792837310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c06cd10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb188009870 tx=0x7fb18802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaed867074772a8a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.835+0000 7fb183fff640 1 -- 172.21.15.111:0/2792837310 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188031640 con 0x7fb18c1544c0 2023-12-04T05:40:41.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.835+0000 7fb183fff640 1 -- 172.21.15.111:0/2792837310 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1880317a0 con 0x7fb18c1544c0 2023-12-04T05:40:41.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.835+0000 7fb183fff640 1 -- 172.21.15.111:0/2792837310 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188031b30 con 0x7fb18c1544c0 2023-12-04T05:40:41.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 -- 172.21.15.111:0/2792837310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 msgr2=0x7fb18c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2792837310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c06cd10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb188009870 tx=0x7fb18802b8f0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 -- 172.21.15.111:0/2792837310 shutdown_connections 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2792837310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c06b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2792837310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c06cd10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 -- 172.21.15.111:0/2792837310 >> 172.21.15.111:0/2792837310 conn(0x7fb18c0b1b50 msgr2=0x7fb18c070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 -- 172.21.15.111:0/2792837310 shutdown_connections 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.836+0000 7fb1938cf640 1 -- 172.21.15.111:0/2792837310 wait complete. 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.837+0000 7fb1938cf640 1 Processor -- start 2023-12-04T05:40:41.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.837+0000 7fb1938cf640 1 -- start start 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c06ba10 0x7fb18c153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c14c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c14c860 con 0x7fb18c1544c0 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c1533f0 con 0x7fb18c06ba10 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c1527e0 con 0x7fb18c06d250 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56044/0 (socket says 172.21.15.111:56044) 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 -- 172.21.15.111:0/2564189818 learned_addr learned my addr 172.21.15.111:0/2564189818 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb190e6a640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c14c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb19166b640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c06ba10 0x7fb18c153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 -- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 msgr2=0x7fb18c14c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c14c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 -- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c06ba10 msgr2=0x7fb18c153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c06ba10 0x7fb18c153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 -- 172.21.15.111:0/2564189818 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1880094d0 con 0x7fb18c1544c0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb191e6c640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c154a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb18400c780 tx=0x7fb18400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184004b60 con 0x7fb18c1544c0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400ce90 con 0x7fb18c1544c0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400a850 con 0x7fb18c1544c0 2023-12-04T05:40:41.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.838+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c152ac0 con 0x7fb18c1544c0 2023-12-04T05:40:41.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.839+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c153080 con 0x7fb18c1544c0 2023-12-04T05:40:41.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.840+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c1535a0 con 0x7fb18c1544c0 2023-12-04T05:40:41.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.840+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb18400a9b0 con 0x7fb18c1544c0 2023-12-04T05:40:41.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.840+0000 7fb181ffb640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb17006b2a0 0x7fb17006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:41.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.840+0000 7fb19166b640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb17006b2a0 0x7fb17006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:41.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.840+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb184089c60 con 0x7fb18c1544c0 2023-12-04T05:40:41.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.841+0000 7fb19166b640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb17006b2a0 0x7fb17006d760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb188036000 tx=0x7fb18803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:41.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:41.845+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb18c1535a0 con 0x7fb18c1544c0 2023-12-04T05:40:42.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.023+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb18c153560 con 0x7fb18c1544c0 2023-12-04T05:40:42.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.024+0000 7fb181ffb640 1 -- 172.21.15.111:0/2564189818 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb184054b80 con 0x7fb18c1544c0 2023-12-04T05:40:42.025 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-12-04T05:40:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb17006b2a0 msgr2=0x7fb17006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb17006b2a0 0x7fb17006d760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb188036000 tx=0x7fb18803c040 comp rx=0 tx=0).stop 2023-12-04T05:40:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 msgr2=0x7fb18c154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c154a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb18400c780 tx=0x7fb18400cc40 comp rx=0 tx=0).stop 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 shutdown_connections 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb17006b2a0 0x7fb17006d760 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb18c1544c0 0x7fb18c154a90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c06d250 0x7fb18c14c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 --2- 172.21.15.111:0/2564189818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c06ba10 0x7fb18c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 >> 172.21.15.111:0/2564189818 conn(0x7fb18c0b1b50 msgr2=0x7fb18c15f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 shutdown_connections 2023-12-04T05:40:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:42.026+0000 7fb1938cf640 1 -- 172.21.15.111:0/2564189818 wait complete. 2023-12-04T05:40:42.038 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-12-04T05:40:42.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:42.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-04T05:40:42.063 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[105312]: fuse finished with error 0 and tester_r 0 2023-12-04T05:40:42.065 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:46.481 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:40:46.484 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:40:48.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:40:48.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:48.166 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-04T05:40:48.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2023-12-04T05:40:48.349 INFO:teuthology.orchestra.run.smithi111.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":70,"msgr_send_messages":65,"msgr_recv_bytes":115583,"msgr_send_bytes":18166,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.011390619,"msgr_running_send_time":0.001744809,"msgr_running_recv_time":0.013052941,"msgr_running_fast_dispatch_time":0.000434041,"msgr_send_messages_queue_lat":{"avgcount":64,"sum":0.044009037,"avgtime":0.000687641},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":113728,"msgr_send_encrypted_bytes":15344},"AsyncMessenger::Worker-1":{"msgr_recv_messages":31,"msgr_send_messages":33,"msgr_recv_bytes":9194,"msgr_send_bytes":126117,"msgr_created_connections":9,"msgr_active_connections":3,"msgr_running_total_time":0.009838441,"msgr_running_send_time":0.001332878,"msgr_running_recv_time":0.007897943,"msgr_running_fast_dispatch_time":0.000423833,"msgr_send_messages_queue_lat":{"avgcount":32,"sum":0.032607884,"avgtime":0.001018996},"msgr_handle_ack_lat":{"avgcount":31,"sum":0.000004355,"avgtime":0.000000140},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":101248},"AsyncMessenger::Worker-2":{"msgr_recv_messages":52,"msgr_send_messages":41,"msgr_recv_bytes":15272,"msgr_send_bytes":25845,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.006411798,"msgr_running_send_time":0.000865976,"msgr_running_recv_time":0.004081519,"msgr_running_fast_dispatch_time":0.001329107,"msgr_send_messages_queue_lat":{"avgcount":40,"sum":0.110347670,"avgtime":0.002758691},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000002995,"avgtime":0.000000142},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":36,"sum":0.006598320,"avgtime":0.000183286}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000012517,"avgtime":0.000006258}},"mds":{"request":4,"reply":4,"reply_latency":{"avgcount":4,"sum":0.000806800,"avgtime":0.000201700},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":0,"caps":0,"subtrees":2,"traverse":7,"traverse_hit":7,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":18,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":9,"evex":0,"evtrm":0,"ev":9,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197804,"rdpos":0,"jlat":{"avgcount":7,"sum":0.010824936,"avgtime":0.001546419},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":3,"cap-":3,"rss":42400,"heap":207196},"mds_server":{"dispatch_client_request":4,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":12,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":3,"sum":0.000566023,"avgtime":0.000188674},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000240777,"avgtime":0.000240777},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":3,"session_remove":3,"sessions_open":0,"sessions_stale":0,"total_load":3,"average_load":3,"avg_session_uptime":1,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":221723,"buffer_anon_items":65,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51520,"mds_co_items":211,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":14710,"op_resend":37,"op_reply":46,"op_latency":{"avgcount":46,"sum":0.590989654,"avgtime":0.012847601},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":70},"op":46,"op_r":0,"op_w":46,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":7,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":33,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":148,"get_sum":122043,"get_or_fail_fail":0,"get_or_fail_success":148,"take":0,"take_sum":0,"put":148,"put_sum":122043,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":14710,"put":46,"put_sum":14710,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5596555866e0":{"val":0,"max":3758096384,"get_started":0,"get":9,"get_sum":3500,"get_or_fail_fail":0,"get_or_fail_success":9,"take":0,"take_sum":0,"put":7,"put_sum":3500,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-12-04T05:40:48.350 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 115583, "msgr_recv_encrypted_bytes": 113728, "msgr_recv_messages": 70, "msgr_running_fast_dispatch_time": 0.000434041, "msgr_running_recv_time": 0.013052941, "msgr_running_send_time": 0.001744809, "msgr_running_total_time": 0.011390619, "msgr_send_bytes": 18166, "msgr_send_encrypted_bytes": 15344, "msgr_send_messages": 65, "msgr_send_messages_queue_lat": { "avgcount": 64, "avgtime": 0.000687641, "sum": 0.044009037 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 31, "avgtime": 1.4e-07, "sum": 4.355e-06 }, "msgr_recv_bytes": 9194, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 31, "msgr_running_fast_dispatch_time": 0.000423833, "msgr_running_recv_time": 0.007897943, "msgr_running_send_time": 0.001332878, "msgr_running_total_time": 0.009838441, "msgr_send_bytes": 126117, "msgr_send_encrypted_bytes": 101248, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 32, "avgtime": 0.001018996, "sum": 0.032607884 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 1.42e-07, "sum": 2.995e-06 }, "msgr_recv_bytes": 15272, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 52, "msgr_running_fast_dispatch_time": 0.001329107, "msgr_running_recv_time": 0.004081519, "msgr_running_send_time": 0.000865976, "msgr_running_total_time": 0.006411798, "msgr_send_bytes": 25845, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 41, "msgr_send_messages_queue_lat": { "avgcount": 40, "avgtime": 0.002758691, "sum": 0.11034767 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 36, "avgtime": 0.000183286, "sum": 0.00659832 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 6.258e-06, "sum": 1.2517e-05 }, "queue_len": 0 }, "mds": { "caps": 0, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 0, "load_cent": 18, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 4, "reply_latency": { "avgcount": 4, "avgtime": 0.0002017, "sum": 0.0008068 }, "request": 4, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 7, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 7, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 9, "evadd": 9, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 7, "avgtime": 0.001546419, "sum": 0.010824936 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197804 }, "mds_mem": { "cap": 0, "cap+": 3, "cap-": 3, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207196, "ino": 13, "ino+": 13, "ino-": 0, "rss": 42400 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 4, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 12, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 3, "avgtime": 0.000188674, "sum": 0.000566023 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000240777, "sum": 0.000240777 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 3, "avg_session_uptime": 1, "mdthresh_evicted": 0, "session_add": 3, "session_count": 0, "session_remove": 3, "sessions_open": 0, "sessions_stale": 0, "total_load": 3 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 221723, "buffer_anon_items": 65, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51520, "mds_co_items": 211, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 33, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.012847601, "sum": 0.590989654 }, "op_pg": 0, "op_r": 0, "op_reply": 46, "op_resend": 37, "op_rmw": 0, "op_send": 46, "op_send_bytes": 14710, "op_w": 46, "oplen_avg": { "avgcount": 46, "sum": 70 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 7, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 148, "get_or_fail_fail": 0, "get_or_fail_success": 148, "get_started": 0, "get_sum": 122043, "max": 104857600, "put": 148, "put_sum": 122043, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 14710, "take": 46, "take_sum": 14710, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5596555866e0": { "get": 9, "get_or_fail_fail": 0, "get_or_fail_success": 9, "get_started": 0, "get_sum": 3500, "max": 3758096384, "put": 7, "put_sum": 3500, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-12-04T05:40:48.350 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-04T05:40:48.351 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-04T05:40:48.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:40:48.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:40:48.441 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-04T05:40:48.441 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:48.441 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:48.456 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:40:48.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:40:48.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:48.457 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-04T05:40:48.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:40:48.511 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-12-04T05:40:48.511 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:40:48.512 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:48.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:40:48.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:40:48.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:48.530 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-04T05:40:48.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:40:48.587 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-12-04T05:40:48.587 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:48.587 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:48.602 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:40:48.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:40:48.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:48.603 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-04T05:40:48.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:40:48.658 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-04T05:40:48.658 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:40:48.658 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:48.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:40:48.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:40:48.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:40:48.675 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-04T05:40:48.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:40:48.732 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-04T05:40:48.732 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.832+0000 7fafce023640 1 Processor -- start 2023-12-04T05:40:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.832+0000 7fafce023640 1 -- start start 2023-12-04T05:40:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafce023640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafc806cae0 0x7fafc806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafce023640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc80b3310 0x7fafc8159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafce023640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fafc815dab0 0x7fafc80703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafce023640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc8154220 con 0x7fafc806cae0 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafce023640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc815d2f0 con 0x7fafc815dab0 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafce023640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc806d400 con 0x7fafc80b3310 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafc806cae0 0x7fafc806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafc806cae0 0x7fafc806cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56050/0 (socket says 172.21.15.111:56050) 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 -- 172.21.15.111:0/2226655369 learned_addr learned my addr 172.21.15.111:0/2226655369 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc6ffd640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc80b3310 0x7fafc8159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc7fff640 1 --2- 172.21.15.111:0/2226655369 >> v2:172.21.15.192:3300/0 conn(0x7fafc815dab0 0x7fafc80703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 -- 172.21.15.111:0/2226655369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc80b3310 msgr2=0x7fafc8159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc80b3310 0x7fafc8159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 -- 172.21.15.111:0/2226655369 >> v2:172.21.15.192:3300/0 conn(0x7fafc815dab0 msgr2=0x7fafc80703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 --2- 172.21.15.111:0/2226655369 >> v2:172.21.15.192:3300/0 conn(0x7fafc815dab0 0x7fafc80703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc77fe640 1 -- 172.21.15.111:0/2226655369 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc815a5c0 con 0x7fafc806cae0 2023-12-04T05:40:48.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.833+0000 7fafc6ffd640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc80b3310 0x7fafc8159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafc77fe640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc806cae0 0x7fafc806cec0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fafc80b3090 tx=0x7fafb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9344da42cdc1828 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafc67fc640 1 -- 172.21.15.111:0/2226655369 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8031870 con 0x7fafc806cae0 2023-12-04T05:40:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafc67fc640 1 -- 172.21.15.111:0/2226655369 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb80319d0 con 0x7fafc806cae0 2023-12-04T05:40:48.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafc67fc640 1 -- 172.21.15.111:0/2226655369 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8031d60 con 0x7fafc806cae0 2023-12-04T05:40:48.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 -- 172.21.15.111:0/2226655369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc806cae0 msgr2=0x7fafc806cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:48.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc806cae0 0x7fafc806cec0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fafc80b3090 tx=0x7fafb802b910 comp rx=0 tx=0).stop 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 -- 172.21.15.111:0/2226655369 shutdown_connections 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 --2- 172.21.15.111:0/2226655369 >> v2:172.21.15.192:3300/0 conn(0x7fafc815dab0 0x7fafc80703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc80b3310 0x7fafc8159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 --2- 172.21.15.111:0/2226655369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc806cae0 0x7fafc806cec0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 -- 172.21.15.111:0/2226655369 >> 172.21.15.111:0/2226655369 conn(0x7fafc806a4c0 msgr2=0x7fafc806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 -- 172.21.15.111:0/2226655369 shutdown_connections 2023-12-04T05:40:48.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.834+0000 7fafce023640 1 -- 172.21.15.111:0/2226655369 wait complete. 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 Processor -- start 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 -- start start 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc806cae0 0x7fafc8149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc80b3310 0x7fafc80ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 0x7fafc8158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81677e0 con 0x7fafc815dab0 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80ad250 con 0x7fafc80b3310 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafce023640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80ad530 con 0x7fafc806cae0 2023-12-04T05:40:48.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.836+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 0x7fafc8158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 0x7fafc8158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56058/0 (socket says 172.21.15.111:56058) 2023-12-04T05:40:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 -- 172.21.15.111:0/2574590352 learned_addr learned my addr 172.21.15.111:0/2574590352 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:48.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc6ffd640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc80b3310 0x7fafc80ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc77fe640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc806cae0 0x7fafc8149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 -- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc806cae0 msgr2=0x7fafc8149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc806cae0 0x7fafc8149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 -- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc80b3310 msgr2=0x7fafc80ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc80b3310 0x7fafc80ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 -- 172.21.15.111:0/2574590352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80094d0 con 0x7fafc815dab0 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc6ffd640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc80b3310 0x7fafc80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafc7fff640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 0x7fafc8158550 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fafc0007e40 tx=0x7fafc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:48.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0014620 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000ee90 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc80ad810 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00149e0 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.837+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc80addd0 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.838+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc80adff0 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.839+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fafc0014b40 con 0x7fafc815dab0 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.839+0000 7fafaffff640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fafb406b2a0 0x7fafb406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:48.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.839+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafc008a5b0 con 0x7fafc815dab0 2023-12-04T05:40:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.839+0000 7fafc77fe640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fafb406b2a0 0x7fafb406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.839+0000 7fafc77fe640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fafb406b2a0 0x7fafb406d760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fafb8036000 tx=0x7fafb803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:48.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:48.843+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fafc00553a0 con 0x7fafc815dab0 2023-12-04T05:40:49.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.046+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafc80693d0 con 0x7fafc815dab0 2023-12-04T05:40:49.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.046+0000 7fafaffff640 1 -- 172.21.15.111:0/2574590352 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fafc00589c0 con 0x7fafc815dab0 2023-12-04T05:40:49.048 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:49.048 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:49.049 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:49.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fafb406b2a0 msgr2=0x7fafb406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fafb406b2a0 0x7fafb406d760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fafb8036000 tx=0x7fafb803c040 comp rx=0 tx=0).stop 2023-12-04T05:40:49.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 msgr2=0x7fafc8158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 0x7fafc8158550 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fafc0007e40 tx=0x7fafc000e4b0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 shutdown_connections 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fafb406b2a0 0x7fafb406d760 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafc815dab0 0x7fafc8158550 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc80b3310 0x7fafc80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 --2- 172.21.15.111:0/2574590352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc806cae0 0x7fafc8149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 >> 172.21.15.111:0/2574590352 conn(0x7fafc806a4c0 msgr2=0x7fafc80cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 shutdown_connections 2023-12-04T05:40:49.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.048+0000 7fafce023640 1 -- 172.21.15.111:0/2574590352 wait complete. 2023-12-04T05:40:49.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.159+0000 7f6fa0b4c640 1 Processor -- start 2023-12-04T05:40:49.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.159+0000 7f6fa0b4c640 1 -- start start 2023-12-04T05:40:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6fa0b4c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f9c0d1780 0x7f6f9c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6fa0b4c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0b20a0 0x7f6f9c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6fa0b4c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c0cdcb0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6fa0b4c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0d20a0 con 0x7f6f9c0d1780 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6fa0b4c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0d39e0 con 0x7f6f9c0b20a0 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6f9a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f9c0d1780 0x7f6f9c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6f9a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f9c0d1780 0x7f6f9c0d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56062/0 (socket says 172.21.15.111:56062) 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.160+0000 7f6f9a59c640 1 -- 172.21.15.111:0/3912796546 learned_addr learned my addr 172.21.15.111:0/3912796546 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.161+0000 7f6f99d9b640 1 --2- 172.21.15.111:0/3912796546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0b20a0 0x7f6f9c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.161+0000 7f6f9a59c640 1 -- 172.21.15.111:0/3912796546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0b20a0 msgr2=0x7f6f9c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.161+0000 7f6f9a59c640 1 --2- 172.21.15.111:0/3912796546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0b20a0 0x7f6f9c0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.161+0000 7f6f9a59c640 1 -- 172.21.15.111:0/3912796546 >> v1:172.21.15.192:6789/0 conn(0x7f6f9c0b2ad0 legacy=0x7f6f9c0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.161+0000 7f6f9a59c640 1 -- 172.21.15.111:0/3912796546 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0d3cc0 con 0x7f6f9c0d1780 2023-12-04T05:40:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.161+0000 7f6f9a59c640 1 --2- 172.21.15.111:0/3912796546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0d1780 0x7f6f9c0d01e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009e50 tx=0x7f6f8c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6265dd6f91c3bde8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6f9959a640 1 -- 172.21.15.111:0/3912796546 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c02ca70 con 0x7f6f9c0d1780 2023-12-04T05:40:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6f9959a640 1 -- 172.21.15.111:0/3912796546 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c02cbd0 con 0x7f6f9c0d1780 2023-12-04T05:40:49.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6f9959a640 1 -- 172.21.15.111:0/3912796546 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0343f0 con 0x7f6f9c0d1780 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/3912796546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0d1780 msgr2=0x7f6f9c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/3912796546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0d1780 0x7f6f9c0d01e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009e50 tx=0x7f6f8c02bed0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/3912796546 shutdown_connections 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/3912796546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0b20a0 0x7f6f9c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/3912796546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0d1780 0x7f6f9c0d01e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/3912796546 >> 172.21.15.111:0/3912796546 conn(0x7f6f9c09c4e0 msgr2=0x7f6f9c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/3912796546 shutdown_connections 2023-12-04T05:40:49.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.162+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/3912796546 wait complete. 2023-12-04T05:40:49.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.163+0000 7f6fa0b4c640 1 Processor -- start 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 -- start start 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f9c0b20a0 0x7f6f9c159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 0x7f6f9c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0d1780 0x7f6f9c0b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1592b0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0ccaa0 con 0x7f6f9c0b20a0 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6fa0b4c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0cac80 con 0x7f6f9c0d1780 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6f99d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 0x7f6f9c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6f99d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 0x7f6f9c158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56078/0 (socket says 172.21.15.111:56078) 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6f99d9b640 1 -- 172.21.15.111:0/1721255100 learned_addr learned my addr 172.21.15.111:0/1721255100 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.164+0000 7f6f9a59c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f9c0b20a0 0x7f6f9c159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f9ad9d640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0d1780 0x7f6f9c0b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f99d9b640 1 -- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0d1780 msgr2=0x7f6f9c0b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f99d9b640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0d1780 0x7f6f9c0b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f99d9b640 1 -- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f9c0b20a0 msgr2=0x7f6f9c159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f99d9b640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f9c0b20a0 0x7f6f9c159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f99d9b640 1 -- 172.21.15.111:0/1721255100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c009b30 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f9ad9d640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0d1780 0x7f6f9c0b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f99d9b640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 0x7f6f9c158d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000d5f0 tx=0x7f6f9000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f900125f0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f90012750 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c0caf60 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90012b00 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.165+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c0cb520 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.166+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c0cb6e0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.166+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6f90012c60 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.167+0000 7f6f8affd640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f7c06b2a0 0x7f6f7c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.167+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f90089fe0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.167+0000 7f6f9a59c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f7c06b2a0 0x7f6f7c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.167+0000 7f6f9a59c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f7c06b2a0 0x7f6f7c06d760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c035000 tx=0x7f6f8c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.171+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f90054ee0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.343+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f9c0693d0 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.344+0000 7f6f8affd640 1 -- 172.21.15.111:0/1721255100 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f6f90058500 con 0x7f6f9c0b2ad0 2023-12-04T05:40:49.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:49.345 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:49.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f7c06b2a0 msgr2=0x7f6f7c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f7c06b2a0 0x7f6f7c06d760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c035000 tx=0x7f6f8c030000 comp rx=0 tx=0).stop 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 msgr2=0x7f6f9c158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 0x7f6f9c158d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000d5f0 tx=0x7f6f9000dab0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 shutdown_connections 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f7c06b2a0 0x7f6f7c06d760 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f9c0d1780 0x7f6f9c0b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f9c0b2ad0 0x7f6f9c158d70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 --2- 172.21.15.111:0/1721255100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f9c0b20a0 0x7f6f9c159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 >> 172.21.15.111:0/1721255100 conn(0x7f6f9c09c4e0 msgr2=0x7f6f9c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 shutdown_connections 2023-12-04T05:40:49.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.346+0000 7f6fa0b4c640 1 -- 172.21.15.111:0/1721255100 wait complete. 2023-12-04T05:40:49.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:40:09.160281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:40:09.699085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:49.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:49.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.461+0000 7f5df771d640 1 Processor -- start 2023-12-04T05:40:49.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.461+0000 7f5df771d640 1 -- start start 2023-12-04T05:40:49.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df771d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5df015b0c0 0x7f5df015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df771d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df00cd320 con 0x7f5df00ce940 2023-12-04T05:40:49.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df771d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df00ced80 con 0x7f5df0072250 2023-12-04T05:40:49.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df771d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df00cc710 con 0x7f5df015b0c0 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df54b9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5df015b0c0 0x7f5df015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df4cb8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5df00ce940 0x7f5df0071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53854/0 (socket says 172.21.15.111:53854) 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df4cb8640 1 -- 172.21.15.111:0/4001026950 learned_addr learned my addr 172.21.15.111:0/4001026950 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df54b9640 1 -- 172.21.15.111:0/4001026950 >> v1:172.21.15.192:6789/0 conn(0x7f5df0072250 legacy=0x7f5df015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df54b9640 1 -- 172.21.15.111:0/4001026950 >> v1:172.21.15.111:6789/0 conn(0x7f5df00ce940 legacy=0x7f5df0071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5df54b9640 1 -- 172.21.15.111:0/4001026950 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df00cc9f0 con 0x7f5df015b0c0 2023-12-04T05:40:49.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.462+0000 7f5deffff640 1 -- 172.21.15.111:0/4001026950 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128304498 0 0) 0x7f5df00cd320 con 0x7f5df00ce940 2023-12-04T05:40:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5df54b9640 1 --2- 172.21.15.111:0/4001026950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015ed10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097a0 tx=0x7f5de802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58081394945c0752 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5deffff640 1 -- 172.21.15.111:0/4001026950 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de802e070 con 0x7f5df015b0c0 2023-12-04T05:40:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5deffff640 1 -- 172.21.15.111:0/4001026950 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80315a0 con 0x7f5df015b0c0 2023-12-04T05:40:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5deffff640 1 -- 172.21.15.111:0/4001026950 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8031930 con 0x7f5df015b0c0 2023-12-04T05:40:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5df771d640 1 -- 172.21.15.111:0/4001026950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 msgr2=0x7f5df015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5df771d640 1 --2- 172.21.15.111:0/4001026950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015ed10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097a0 tx=0x7f5de802b840 comp rx=0 tx=0).stop 2023-12-04T05:40:49.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5df771d640 1 -- 172.21.15.111:0/4001026950 shutdown_connections 2023-12-04T05:40:49.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5df771d640 1 --2- 172.21.15.111:0/4001026950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015ed10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.463+0000 7f5df771d640 1 -- 172.21.15.111:0/4001026950 >> 172.21.15.111:0/4001026950 conn(0x7f5df00b0cf0 msgr2=0x7f5df00b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.464+0000 7f5df771d640 1 -- 172.21.15.111:0/4001026950 shutdown_connections 2023-12-04T05:40:49.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.464+0000 7f5df771d640 1 -- 172.21.15.111:0/4001026950 wait complete. 2023-12-04T05:40:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 Processor -- start 2023-12-04T05:40:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 -- start start 2023-12-04T05:40:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df0072250 0x7f5df0145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5df00ce940 0x7f5df0159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01f53a0 con 0x7f5df015b0c0 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01f1d90 con 0x7f5df0072250 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df771d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0145750 con 0x7f5df00ce940 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df5cba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df5cba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56090/0 (socket says 172.21.15.111:56090) 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.465+0000 7f5df5cba640 1 -- 172.21.15.111:0/856728152 learned_addr learned my addr 172.21.15.111:0/856728152 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df54b9640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df0072250 0x7f5df0145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df4cb8640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5df00ce940 0x7f5df0159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df5cba640 1 -- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5df00ce940 msgr2=0x7f5df0159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df5cba640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5df00ce940 0x7f5df0159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df5cba640 1 -- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df0072250 msgr2=0x7f5df0145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df5cba640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df0072250 0x7f5df0145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df5cba640 1 -- 172.21.15.111:0/856728152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8009400 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df4cb8640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5df00ce940 0x7f5df0159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df5cba640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015a180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c7e0 tx=0x7f5de000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0003d80 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0005070 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0005420 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df015b620 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.466+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df01f0d60 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.467+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df0069360 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.468+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5de0005580 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.468+0000 7f5dedffb640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5ddc06b2a0 0x7f5ddc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.468+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de0088e90 con 0x7f5df015b0c0 2023-12-04T05:40:49.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.468+0000 7f5df54b9640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5ddc06b2a0 0x7f5ddc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.469+0000 7f5df54b9640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5ddc06b2a0 0x7f5ddc06d760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5de8036000 tx=0x7f5de803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.472+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5de0053d90 con 0x7f5df015b0c0 2023-12-04T05:40:49.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.641+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5df00aa560 con 0x7f5df015b0c0 2023-12-04T05:40:49.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.641+0000 7f5dedffb640 1 -- 172.21.15.111:0/856728152 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5de00573b0 con 0x7f5df015b0c0 2023-12-04T05:40:49.643 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5ddc06b2a0 msgr2=0x7f5ddc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5ddc06b2a0 0x7f5ddc06d760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5de8036000 tx=0x7f5de803d040 comp rx=0 tx=0).stop 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 msgr2=0x7f5df015a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015a180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c7e0 tx=0x7f5de000cca0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 shutdown_connections 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f5ddc06b2a0 0x7f5ddc06d760 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5df015b0c0 0x7f5df015a180 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5df00ce940 0x7f5df0159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 --2- 172.21.15.111:0/856728152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5df0072250 0x7f5df0145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 >> 172.21.15.111:0/856728152 conn(0x7f5df00b0cf0 msgr2=0x7f5df0155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 shutdown_connections 2023-12-04T05:40:49.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.643+0000 7f5df771d640 1 -- 172.21.15.111:0/856728152 wait complete. 2023-12-04T05:40:49.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:40:49.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.756+0000 7f1ee8168640 1 Processor -- start 2023-12-04T05:40:49.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.756+0000 7f1ee8168640 1 -- start start 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee8168640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1ee0160920 0x7f1ee006d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee8168640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ee006b4d0 0x7f1ee006b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee8168640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b46a0 con 0x7f1ee00b3a20 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee8168640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00b3850 con 0x7f1ee006b4d0 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee8168640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee006bea0 con 0x7f1ee0160920 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee5703640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ee006b4d0 0x7f1ee006b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee5703640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1ee006b4d0 0x7f1ee006b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56104/0 (socket says 172.21.15.111:56104) 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.757+0000 7f1ee5703640 1 -- 172.21.15.111:0/2215936183 learned_addr learned my addr 172.21.15.111:0/2215936183 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5f04640 1 --2- 172.21.15.111:0/2215936183 >> v2:172.21.15.192:3300/0 conn(0x7f1ee0160920 0x7f1ee006d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5703640 1 -- 172.21.15.111:0/2215936183 >> v2:172.21.15.192:3300/0 conn(0x7f1ee0160920 msgr2=0x7f1ee006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5703640 1 --2- 172.21.15.111:0/2215936183 >> v2:172.21.15.192:3300/0 conn(0x7f1ee0160920 0x7f1ee006d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5703640 1 -- 172.21.15.111:0/2215936183 >> v1:172.21.15.192:6789/0 conn(0x7f1ee00b3a20 legacy=0x7f1ee00b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5703640 1 -- 172.21.15.111:0/2215936183 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee006cb60 con 0x7f1ee006b4d0 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5f04640 1 --2- 172.21.15.111:0/2215936183 >> v2:172.21.15.192:3300/0 conn(0x7f1ee0160920 0x7f1ee006d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee5703640 1 --2- 172.21.15.111:0/2215936183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee006b960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0071550 tx=0x7f1ed002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23e7f9e41626777 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee4f02640 1 -- 172.21.15.111:0/2215936183 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0031870 con 0x7f1ee006b4d0 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.758+0000 7f1ee4f02640 1 -- 172.21.15.111:0/2215936183 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed00319d0 con 0x7f1ee006b4d0 2023-12-04T05:40:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee4f02640 1 -- 172.21.15.111:0/2215936183 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0031d60 con 0x7f1ee006b4d0 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 -- 172.21.15.111:0/2215936183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 msgr2=0x7f1ee006b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 --2- 172.21.15.111:0/2215936183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee006b960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0071550 tx=0x7f1ed002b910 comp rx=0 tx=0).stop 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 -- 172.21.15.111:0/2215936183 shutdown_connections 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 --2- 172.21.15.111:0/2215936183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee006b960 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 --2- 172.21.15.111:0/2215936183 >> v2:172.21.15.192:3300/0 conn(0x7f1ee0160920 0x7f1ee006d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 -- 172.21.15.111:0/2215936183 >> 172.21.15.111:0/2215936183 conn(0x7f1ee00b1b90 msgr2=0x7f1ee0070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 -- 172.21.15.111:0/2215936183 shutdown_connections 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.759+0000 7f1ee8168640 1 -- 172.21.15.111:0/2215936183 wait complete. 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.760+0000 7f1ee8168640 1 Processor -- start 2023-12-04T05:40:49.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.760+0000 7f1ee8168640 1 -- start start 2023-12-04T05:40:49.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee8168640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee0149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee8168640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ee00b3a20 0x7f1ee0158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee8168640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ee0160920 0x7f1ee0149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee8168640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01496c0 con 0x7f1ee006b4d0 2023-12-04T05:40:49.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee8168640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01581c0 con 0x7f1ee0160920 2023-12-04T05:40:49.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee8168640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01575b0 con 0x7f1ee00b3a20 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee0149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee0149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56106/0 (socket says 172.21.15.111:56106) 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 -- 172.21.15.111:0/4167992186 learned_addr learned my addr 172.21.15.111:0/4167992186 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 -- 172.21.15.111:0/4167992186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ee00b3a20 msgr2=0x7f1ee0158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ee00b3a20 0x7f1ee0158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 -- 172.21.15.111:0/4167992186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ee0160920 msgr2=0x7f1ee0149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ee0160920 0x7f1ee0149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 -- 172.21.15.111:0/4167992186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed00094d0 con 0x7f1ee006b4d0 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.761+0000 7f1ee5f04640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee0149180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00ce00 tx=0x7f1edc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.762+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00f890 con 0x7f1ee006b4d0 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.762+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00f9f0 con 0x7f1ee006b4d0 2023-12-04T05:40:49.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.762+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee0157890 con 0x7f1ee006b4d0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.762+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00fdc0 con 0x7f1ee006b4d0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.762+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0157e50 con 0x7f1ee006b4d0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.763+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee0069360 con 0x7f1ee006b4d0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.763+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1edc00b040 con 0x7f1ee006b4d0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.763+0000 7f1ed67fc640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec406b2a0 0x7f1ec406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.764+0000 7f1ee5703640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec406b2a0 0x7f1ec406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.764+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1edc08a590 con 0x7f1ee006b4d0 2023-12-04T05:40:49.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.764+0000 7f1ee5703640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec406b2a0 0x7f1ec406d760 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0036000 tx=0x7f1ed0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:49.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.768+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1edc055540 con 0x7f1ee006b4d0 2023-12-04T05:40:49.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.936+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ee0071ef0 con 0x7f1ee006b4d0 2023-12-04T05:40:49.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.936+0000 7f1ed67fc640 1 -- 172.21.15.111:0/4167992186 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1edc058b60 con 0x7f1ee006b4d0 2023-12-04T05:40:49.938 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec406b2a0 msgr2=0x7f1ec406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec406b2a0 0x7f1ec406d760 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0036000 tx=0x7f1ed0030000 comp rx=0 tx=0).stop 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 msgr2=0x7f1ee0149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee0149180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00ce00 tx=0x7f1edc00a650 comp rx=0 tx=0).stop 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 shutdown_connections 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ec406b2a0 0x7f1ec406d760 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ee0160920 0x7f1ee0149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ee00b3a20 0x7f1ee0158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 --2- 172.21.15.111:0/4167992186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ee006b4d0 0x7f1ee0149180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:49.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 >> 172.21.15.111:0/4167992186 conn(0x7f1ee00b1b90 msgr2=0x7f1ee015f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:49.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 shutdown_connections 2023-12-04T05:40:49.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:49.938+0000 7f1ee8168640 1 -- 172.21.15.111:0/4167992186 wait complete. 2023-12-04T05:40:49.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:40:50.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.051+0000 7f5154590640 1 Processor -- start 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.052+0000 7f5154590640 1 -- start start 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.052+0000 7f5154590640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 0x7f514c161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.052+0000 7f5154590640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c06eaf0 con 0x7f514c0d0f10 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.052+0000 7f5154590640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c15e4e0 con 0x7f514c15dfb0 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.052+0000 7f5154590640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c15e7c0 con 0x7f514c146b10 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f5151b2b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f514c0d0f10 0x7f514c15d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53868/0 (socket says 172.21.15.111:53868) 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f5151b2b640 1 -- 172.21.15.111:0/3375020339 learned_addr learned my addr 172.21.15.111:0/3375020339 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515232c640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 0x7f514c161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732357853 0 0) 0x7f514c06eaf0 con 0x7f514c0d0f10 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5138003520 con 0x7f514c0d0f10 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15940062 0 0) 0x7f5138003520 con 0x7f514c0d0f10 2023-12-04T05:40:50.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 msgr2=0x7f514c161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 0x7f514c161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 >> v1:172.21.15.192:6789/0 conn(0x7f514c15dfb0 legacy=0x7f514c06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c160f50 con 0x7f514c0d0f10 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f513c001670 con 0x7f514c0d0f10 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 >> v1:172.21.15.111:6789/0 conn(0x7f514c0d0f10 legacy=0x7f514c15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51380039a0 0x7f5138003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f5138004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 0x7f5138005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c15e7c0 con 0x7f5138004350 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513800b980 con 0x7f51380039a0 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513800bb40 con 0x7f514c146b10 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f5151b2b640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f5138004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f515232c640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51380039a0 0x7f5138003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.053+0000 7f5152b2d640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 0x7f5138005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f5151b2b640 1 -- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 msgr2=0x7f5138005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f5151b2b640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c146b10 0x7f5138005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f5151b2b640 1 -- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51380039a0 msgr2=0x7f5138003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f5151b2b640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51380039a0 0x7f5138003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f5151b2b640 1 -- 172.21.15.111:0/3375020339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138003520 con 0x7f5138004350 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f5151b2b640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f5138004820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c240 tx=0x7f51380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d8298da5c1bc22 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.054+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c005ad0 con 0x7f5138004350 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c005c30 con 0x7f5138004350 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f515132a640 1 -- 172.21.15.111:0/3375020339 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c02c8a0 con 0x7f5138004350 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5154590640 1 -- 172.21.15.111:0/3375020339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 msgr2=0x7f5138004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5154590640 1 --2- 172.21.15.111:0/3375020339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f5138004820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c240 tx=0x7f51380057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5152b2d640 1 -- 172.21.15.111:0/3375020339 reap_dead start 2023-12-04T05:40:50.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5154590640 1 -- 172.21.15.111:0/3375020339 shutdown_connections 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5154590640 1 -- 172.21.15.111:0/3375020339 >> 172.21.15.111:0/3375020339 conn(0x7f514c070af0 msgr2=0x7f514c0662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5154590640 1 -- 172.21.15.111:0/3375020339 shutdown_connections 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.055+0000 7f5154590640 1 -- 172.21.15.111:0/3375020339 wait complete. 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.056+0000 7f5154590640 1 Processor -- start 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.056+0000 7f5154590640 1 -- start start 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5154590640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f514c13cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5154590640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c15ccf0 0x7f514c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5154590640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c1f3450 0x7f514c13a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5154590640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c141820 con 0x7f5138004350 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5154590640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c141fd0 con 0x7f514c15ccf0 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5154590640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c141a50 con 0x7f514c1f3450 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f514c13cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f514c13cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56120/0 (socket says 172.21.15.111:56120) 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 -- 172.21.15.111:0/1331531537 learned_addr learned my addr 172.21.15.111:0/1331531537 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:50.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5152b2d640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c1f3450 0x7f514c13a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5151b2b640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c15ccf0 0x7f514c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 -- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c1f3450 msgr2=0x7f514c13a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c1f3450 0x7f514c13a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 -- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c15ccf0 msgr2=0x7f514c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c15ccf0 0x7f514c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 -- 172.21.15.111:0/1331531537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c13d8a0 con 0x7f5138004350 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5152b2d640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c1f3450 0x7f514c13a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5151b2b640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c15ccf0 0x7f514c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f515232c640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f514c13cde0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5144009f90 tx=0x7f514400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.057+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400e040 con 0x7f5138004350 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.058+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514400f040 con 0x7f5138004350 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.058+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c141cf0 con 0x7f5138004350 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.058+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144015870 con 0x7f5138004350 2023-12-04T05:40:50.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.058+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c13d320 con 0x7f5138004350 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.059+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c13d570 con 0x7f5138004350 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.059+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f51440159d0 con 0x7f5138004350 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.060+0000 7f5142ffd640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f513006b2d0 0x7f513006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.060+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f514408a9a0 con 0x7f5138004350 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.060+0000 7f5151b2b640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f513006b2d0 0x7f513006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.060+0000 7f5151b2b640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f513006b2d0 0x7f513006d790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c240 tx=0x7f513c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.064+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5144055810 con 0x7f5138004350 2023-12-04T05:40:50.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.230+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f514c069360 con 0x7f5138004350 2023-12-04T05:40:50.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.230+0000 7f5142ffd640 1 -- 172.21.15.111:0/1331531537 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f514400fea0 con 0x7f5138004350 2023-12-04T05:40:50.232 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:40:50.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f513006b2d0 msgr2=0x7f513006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f513006b2d0 0x7f513006d790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c240 tx=0x7f513c009030 comp rx=0 tx=0).stop 2023-12-04T05:40:50.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 msgr2=0x7f514c13cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f514c13cde0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5144009f90 tx=0x7f514400f580 comp rx=0 tx=0).stop 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 shutdown_connections 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f513006b2d0 0x7f513006d790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f514c1f3450 0x7f514c13a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f514c15ccf0 0x7f514c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 --2- 172.21.15.111:0/1331531537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138004350 0x7f514c13cde0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 >> 172.21.15.111:0/1331531537 conn(0x7f514c070af0 msgr2=0x7f514c15f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 shutdown_connections 2023-12-04T05:40:50.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.232+0000 7f5154590640 1 -- 172.21.15.111:0/1331531537 wait complete. 2023-12-04T05:40:50.245 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:50.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.344+0000 7f303631d640 1 Processor -- start 2023-12-04T05:40:50.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.344+0000 7f303631d640 1 -- start start 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f303631d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 0x7f303015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f303631d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3030140200 0x7f303015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f303631d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30300aea20 con 0x7f303013cee0 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f303631d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303013d2c0 con 0x7f3030140200 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f303631d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303015b3a0 con 0x7f303006c960 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f302f7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 0x7f303015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f30348ba640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3030140200 0x7f303015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f302f7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 0x7f303015a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:35324/0 (socket says 172.21.15.111:35324) 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f302f7fe640 1 -- 172.21.15.111:0/1138338595 learned_addr learned my addr 172.21.15.111:0/1138338595 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.345+0000 7f302ffff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f303013cee0 0x7f303006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:42506/0 (socket says 172.21.15.111:42506) 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.346+0000 7f302f7fe640 1 -- 172.21.15.111:0/1138338595 >> v2:172.21.15.192:3300/0 conn(0x7f3030140200 msgr2=0x7f303015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.346+0000 7f302f7fe640 1 --2- 172.21.15.111:0/1138338595 >> v2:172.21.15.192:3300/0 conn(0x7f3030140200 0x7f303015a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.346+0000 7f302f7fe640 1 -- 172.21.15.111:0/1138338595 >> v1:172.21.15.192:6789/0 conn(0x7f303013cee0 legacy=0x7f303006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.346+0000 7f302f7fe640 1 -- 172.21.15.111:0/1138338595 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303015b0c0 con 0x7f303006c960 2023-12-04T05:40:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.346+0000 7f302f7fe640 1 --2- 172.21.15.111:0/1138338595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 0x7f303015a490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3024009870 tx=0x7f302402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ed060b489bfe12 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f302effd640 1 -- 172.21.15.111:0/1138338595 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024031640 con 0x7f303006c960 2023-12-04T05:40:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f302effd640 1 -- 172.21.15.111:0/1138338595 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30240317a0 con 0x7f303006c960 2023-12-04T05:40:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f302effd640 1 -- 172.21.15.111:0/1138338595 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024031b30 con 0x7f303006c960 2023-12-04T05:40:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 -- 172.21.15.111:0/1138338595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 msgr2=0x7f303015a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 --2- 172.21.15.111:0/1138338595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 0x7f303015a490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3024009870 tx=0x7f302402b8f0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 -- 172.21.15.111:0/1138338595 shutdown_connections 2023-12-04T05:40:50.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 --2- 172.21.15.111:0/1138338595 >> v2:172.21.15.192:3300/0 conn(0x7f3030140200 0x7f303015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 --2- 172.21.15.111:0/1138338595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303006c960 0x7f303015a490 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 -- 172.21.15.111:0/1138338595 >> 172.21.15.111:0/1138338595 conn(0x7f303006a4c0 msgr2=0x7f303006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 -- 172.21.15.111:0/1138338595 shutdown_connections 2023-12-04T05:40:50.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.347+0000 7f303631d640 1 -- 172.21.15.111:0/1138338595 wait complete. 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.348+0000 7f303631d640 1 Processor -- start 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 -- start start 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 0x7f30301368a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303013cee0 0x7f303014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030140200 0x7f303014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3030136de0 con 0x7f303006c960 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30300b3010 con 0x7f3030140200 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f303631d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30300b3e50 con 0x7f303013cee0 2023-12-04T05:40:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f302ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 0x7f30301368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f302ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 0x7f30301368a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56134/0 (socket says 172.21.15.111:56134) 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f302ffff640 1 -- 172.21.15.111:0/3656939193 learned_addr learned my addr 172.21.15.111:0/3656939193 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f302f7fe640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303013cee0 0x7f303014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.349+0000 7f30348ba640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030140200 0x7f303014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302ffff640 1 -- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303013cee0 msgr2=0x7f303014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302ffff640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303013cee0 0x7f303014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302ffff640 1 -- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030140200 msgr2=0x7f303014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302ffff640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030140200 0x7f303014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302ffff640 1 -- 172.21.15.111:0/3656939193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240094d0 con 0x7f303006c960 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302ffff640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 0x7f30301368a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f302000c500 tx=0x7f302000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020007500 con 0x7f303006c960 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30300b42e0 con 0x7f303006c960 2023-12-04T05:40:50.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020007660 con 0x7f303006c960 2023-12-04T05:40:50.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30200049e0 con 0x7f303006c960 2023-12-04T05:40:50.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.350+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30300b3520 con 0x7f303006c960 2023-12-04T05:40:50.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.351+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30300693d0 con 0x7f303006c960 2023-12-04T05:40:50.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.351+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3020004c00 con 0x7f303006c960 2023-12-04T05:40:50.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.352+0000 7f302cff9640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f301c06b2a0 0x7f301c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.352+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3020088d60 con 0x7f303006c960 2023-12-04T05:40:50.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.352+0000 7f302f7fe640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f301c06b2a0 0x7f301c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.352+0000 7f302f7fe640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f301c06b2a0 0x7f301c06d760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3024036000 tx=0x7f302403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.356+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3020053c90 con 0x7f303006c960 2023-12-04T05:40:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.559+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f303006d170 con 0x7f303006c960 2023-12-04T05:40:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.559+0000 7f302cff9640 1 -- 172.21.15.111:0/3656939193 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f30200572b0 con 0x7f303006c960 2023-12-04T05:40:50.561 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:50.561 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f301c06b2a0 msgr2=0x7f301c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f301c06b2a0 0x7f301c06d760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3024036000 tx=0x7f302403c040 comp rx=0 tx=0).stop 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 msgr2=0x7f30301368a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 0x7f30301368a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f302000c500 tx=0x7f302000c9c0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 shutdown_connections 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f301c06b2a0 0x7f301c06d760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030140200 0x7f303014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303013cee0 0x7f303014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 --2- 172.21.15.111:0/3656939193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f303006c960 0x7f30301368a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 >> 172.21.15.111:0/3656939193 conn(0x7f303006a4c0 msgr2=0x7f30300703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 shutdown_connections 2023-12-04T05:40:50.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.561+0000 7f303631d640 1 -- 172.21.15.111:0/3656939193 wait complete. 2023-12-04T05:40:50.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:50.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.672+0000 7ffa05d51640 1 Processor -- start 2023-12-04T05:40:50.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.672+0000 7ffa05d51640 1 -- start start 2023-12-04T05:40:50.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ffa05d51640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffa000d01e0 0x7ffa000ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ffa05d51640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000cdcb0 con 0x7ffa000d1780 2023-12-04T05:40:50.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ffa05d51640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000d39e0 con 0x7ffa000b20a0 2023-12-04T05:40:50.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ffa05d51640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000d05f0 con 0x7ffa000d01e0 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ff9fffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffa000d01e0 0x7ffa000ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ff9ff7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ffa000d1780 0x7ffa000d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36928/0 (socket says 172.21.15.111:36928) 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ff9ff7fe640 1 -- 172.21.15.111:0/3135592268 learned_addr learned my addr 172.21.15.111:0/3135592268 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.673+0000 7ff9fffff640 1 --2- 172.21.15.111:0/3135592268 >> v2:172.21.15.111:3300/0 conn(0x7ffa000d01e0 0x7ffa000ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43924/0 (socket says 172.21.15.111:43924) 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.674+0000 7ff9fffff640 1 -- 172.21.15.111:0/3135592268 >> v1:172.21.15.192:6789/0 conn(0x7ffa000b20a0 legacy=0x7ffa000b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.674+0000 7ff9fffff640 1 -- 172.21.15.111:0/3135592268 >> v1:172.21.15.111:6789/0 conn(0x7ffa000d1780 legacy=0x7ffa000d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.674+0000 7ff9fffff640 1 -- 172.21.15.111:0/3135592268 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000d3cc0 con 0x7ffa000d01e0 2023-12-04T05:40:50.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.674+0000 7ff9fe7fc640 1 -- 172.21.15.111:0/3135592268 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633864844 0 0) 0x7ffa000cdcb0 con 0x7ffa000d1780 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.674+0000 7ff9fffff640 1 --2- 172.21.15.111:0/3135592268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa000ce750 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009d90 tx=0x7ff9f802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ae18ddb10d3173 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ff9fe7fc640 1 -- 172.21.15.111:0/3135592268 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f802cbd0 con 0x7ffa000d01e0 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ff9fe7fc640 1 -- 172.21.15.111:0/3135592268 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f802cd30 con 0x7ffa000d01e0 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ff9fe7fc640 1 -- 172.21.15.111:0/3135592268 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f802cea0 con 0x7ffa000d01e0 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 -- 172.21.15.111:0/3135592268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 msgr2=0x7ffa000ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3135592268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa000ce750 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009d90 tx=0x7ff9f802be20 comp rx=0 tx=0).stop 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 -- 172.21.15.111:0/3135592268 shutdown_connections 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3135592268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa000ce750 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 -- 172.21.15.111:0/3135592268 >> 172.21.15.111:0/3135592268 conn(0x7ffa0009c4e0 msgr2=0x7ffa000d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 -- 172.21.15.111:0/3135592268 shutdown_connections 2023-12-04T05:40:50.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.675+0000 7ffa05d51640 1 -- 172.21.15.111:0/3135592268 wait complete. 2023-12-04T05:40:50.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.676+0000 7ffa05d51640 1 Processor -- start 2023-12-04T05:40:50.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.676+0000 7ffa05d51640 1 -- start start 2023-12-04T05:40:50.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ffa05d51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa000b20a0 0x7ffa000b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ffa05d51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa0014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ffa05d51640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa000d1780 0x7ffa000c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ffa05d51640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000b7b30 con 0x7ffa000d01e0 2023-12-04T05:40:50.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ffa05d51640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000b6f20 con 0x7ffa000d1780 2023-12-04T05:40:50.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ffa05d51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000b7200 con 0x7ffa000b20a0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa0014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa0014d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43932/0 (socket says 172.21.15.111:43932) 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 -- 172.21.15.111:0/3532949316 learned_addr learned my addr 172.21.15.111:0/3532949316 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 -- 172.21.15.111:0/3532949316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa000b20a0 msgr2=0x7ffa000b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa000b20a0 0x7ffa000b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 -- 172.21.15.111:0/3532949316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa000d1780 msgr2=0x7ffa000c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa000d1780 0x7ffa000c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.677+0000 7ff9feffd640 1 -- 172.21.15.111:0/3532949316 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8009a70 con 0x7ffa000d01e0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.678+0000 7ff9feffd640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa0014d4b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400b3e0 tx=0x7ff9f400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.678+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4011830 con 0x7ffa000d01e0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.678+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f40119b0 con 0x7ffa000d01e0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.678+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4011d60 con 0x7ffa000d01e0 2023-12-04T05:40:50.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.678+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa000b74e0 con 0x7ffa000d01e0 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.678+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0014d9f0 con 0x7ffa000d01e0 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.679+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa000693d0 con 0x7ffa000d01e0 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.679+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ff9f401a400 con 0x7ffa000d01e0 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.680+0000 7ff9e7fff640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9ec06b2a0 0x7ff9ec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.680+0000 7ff9ff7fe640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9ec06b2a0 0x7ff9ec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.680+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9f4089d50 con 0x7ffa000d01e0 2023-12-04T05:40:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.680+0000 7ff9ff7fe640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9ec06b2a0 0x7ff9ec06d760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0007fd0 tx=0x7ff9f0007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.684+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9f408d030 con 0x7ffa000d01e0 2023-12-04T05:40:50.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.852+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa000d40b0 con 0x7ffa000d01e0 2023-12-04T05:40:50.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.853+0000 7ff9e7fff640 1 -- 172.21.15.111:0/3532949316 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7ff9f4015090 con 0x7ffa000d01e0 2023-12-04T05:40:50.854 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:50.855 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:50.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9ec06b2a0 msgr2=0x7ff9ec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9ec06b2a0 0x7ff9ec06d760 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0007fd0 tx=0x7ff9f0007960 comp rx=0 tx=0).stop 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 msgr2=0x7ffa0014d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa0014d4b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400b3e0 tx=0x7ff9f400b8a0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 shutdown_connections 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ff9ec06b2a0 0x7ff9ec06d760 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa000d1780 0x7ffa000c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa000d01e0 0x7ffa0014d4b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 --2- 172.21.15.111:0/3532949316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa000b20a0 0x7ffa000b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 >> 172.21.15.111:0/3532949316 conn(0x7ffa0009c4e0 msgr2=0x7ffa000b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 shutdown_connections 2023-12-04T05:40:50.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.855+0000 7ffa05d51640 1 -- 172.21.15.111:0/3532949316 wait complete. 2023-12-04T05:40:50.870 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-04T05:40:50.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:50.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.970+0000 7f50af108640 1 Processor -- start 2023-12-04T05:40:50.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.970+0000 7f50af108640 1 -- start start 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50af108640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f50a806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50af108640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a8154220 con 0x7f50a815dab0 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50af108640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a815d2f0 con 0x7f50a80b3310 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50af108640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815a3c0 con 0x7f50a806cae0 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50ad6a5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f50a815dab0 0x7f50a80703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36940/0 (socket says 172.21.15.111:36940) 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50ad6a5640 1 -- 172.21.15.111:0/2592609598 learned_addr learned my addr 172.21.15.111:0/2592609598 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50acea4640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f50a806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452853618 0 0) 0x7f50a0001490 con 0x7f50a815dab0 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5098003520 con 0x7f50a815dab0 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569159835 0 0) 0x7f5098003520 con 0x7f50a815dab0 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 msgr2=0x7f50a806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f50a806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 >> v1:172.21.15.192:6789/0 conn(0x7f50a80b3310 legacy=0x7f50a8159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a806d400 con 0x7f50a815dab0 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f50a0001b80 con 0x7f50a815dab0 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 >> v1:172.21.15.111:6789/0 conn(0x7f50a815dab0 legacy=0x7f50a80703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50980039a0 0x7f5098003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5098004350 0x7f5098004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f5098005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815a3c0 con 0x7f5098004350 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509800b980 con 0x7f50980039a0 2023-12-04T05:40:50.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.971+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509800bb40 con 0x7f50a806cae0 2023-12-04T05:40:50.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5098004350 0x7f5098004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50acea4640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50980039a0 0x7f5098003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50ad6a5640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f5098005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50a7fff640 1 -- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 msgr2=0x7f5098005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f5098005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50a7fff640 1 -- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50980039a0 msgr2=0x7f5098003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50980039a0 0x7f5098003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50a7fff640 1 -- 172.21.15.111:0/2592609598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098003520 con 0x7f5098004350 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.972+0000 7f50ad6a5640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50a806cae0 0x7f5098005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5098004350 0x7f5098004820 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5094001860 tx=0x7f50980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a01e21c7b4154a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094033410 con 0x7f5098004350 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094033590 con 0x7f5098004350 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50a77fe640 1 -- 172.21.15.111:0/2592609598 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094033940 con 0x7f5098004350 2023-12-04T05:40:50.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50af108640 1 -- 172.21.15.111:0/2592609598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5098004350 msgr2=0x7f5098004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50af108640 1 --2- 172.21.15.111:0/2592609598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5098004350 0x7f5098004820 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5094001860 tx=0x7f50980057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50ad6a5640 1 -- 172.21.15.111:0/2592609598 reap_dead start 2023-12-04T05:40:50.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50af108640 1 -- 172.21.15.111:0/2592609598 shutdown_connections 2023-12-04T05:40:50.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50af108640 1 -- 172.21.15.111:0/2592609598 >> 172.21.15.111:0/2592609598 conn(0x7f50a806a4c0 msgr2=0x7f50a806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:50.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50af108640 1 -- 172.21.15.111:0/2592609598 shutdown_connections 2023-12-04T05:40:50.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.973+0000 7f50af108640 1 -- 172.21.15.111:0/2592609598 wait complete. 2023-12-04T05:40:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.974+0000 7f50af108640 1 Processor -- start 2023-12-04T05:40:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.974+0000 7f50af108640 1 -- start start 2023-12-04T05:40:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50af108640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5098004350 0x7f50a8148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50af108640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 0x7f50a81543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50af108640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8156f50 0x7f50a80703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50af108640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81eead0 con 0x7f50a8150700 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50af108640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81eedb0 con 0x7f50a8156f50 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50af108640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81ef090 con 0x7f5098004350 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50a7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 0x7f50a81543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50a7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 0x7f50a81543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43948/0 (socket says 172.21.15.111:43948) 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50a7fff640 1 -- 172.21.15.111:0/2338810791 learned_addr learned my addr 172.21.15.111:0/2338810791 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50acea4640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5098004350 0x7f50a8148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.975+0000 7f50ad6a5640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8156f50 0x7f50a80703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a7fff640 1 -- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5098004350 msgr2=0x7f50a8148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5098004350 0x7f50a8148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a7fff640 1 -- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8156f50 msgr2=0x7f50a80703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8156f50 0x7f50a80703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a7fff640 1 -- 172.21.15.111:0/2338810791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a81ef370 con 0x7f50a8150700 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50ad6a5640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8156f50 0x7f50a80703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a7fff640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 0x7f50a81543c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5094001980 tx=0x7f5094001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940076f0 con 0x7f50a8150700 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094033410 con 0x7f50a8150700 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940052b0 con 0x7f50a8150700 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a8149190 con 0x7f50a8150700 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.976+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a8154900 con 0x7f50a8150700 2023-12-04T05:40:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.977+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a8154af0 con 0x7f50a8150700 2023-12-04T05:40:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.977+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f509403a050 con 0x7f50a8150700 2023-12-04T05:40:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.978+0000 7f50a57fa640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f508c06b230 0x7f508c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.978+0000 7f50acea4640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f508c06b230 0x7f508c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.978+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50940ab580 con 0x7f50a8150700 2023-12-04T05:40:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.978+0000 7f50acea4640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f508c06b230 0x7f508c06d6f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f509c009b00 tx=0x7f509c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:50.982+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50940765c0 con 0x7f50a8150700 2023-12-04T05:40:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.183+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50a80693d0 con 0x7f50a8150700 2023-12-04T05:40:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.183+0000 7f50a57fa640 1 -- 172.21.15.111:0/2338810791 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f5094079be0 con 0x7f50a8150700 2023-12-04T05:40:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:51.185 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f508c06b230 msgr2=0x7f508c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f508c06b230 0x7f508c06d6f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f509c009b00 tx=0x7f509c008040 comp rx=0 tx=0).stop 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 msgr2=0x7f50a81543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 0x7f50a81543c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5094001980 tx=0x7f5094001230 comp rx=0 tx=0).stop 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 shutdown_connections 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f508c06b230 0x7f508c06d6f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50a8156f50 0x7f50a80703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50a8150700 0x7f50a81543c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 --2- 172.21.15.111:0/2338810791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5098004350 0x7f50a8148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 >> 172.21.15.111:0/2338810791 conn(0x7f50a806a4c0 msgr2=0x7f50a80b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 shutdown_connections 2023-12-04T05:40:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.185+0000 7f50af108640 1 -- 172.21.15.111:0/2338810791 wait complete. 2023-12-04T05:40:51.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.296+0000 7f3e12ce5640 1 Processor -- start 2023-12-04T05:40:51.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.296+0000 7f3e12ce5640 1 -- start start 2023-12-04T05:40:51.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e12ce5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3e0c15d4b0 0x7f3e0c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e12ce5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e12ce5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0c0d2d80 con 0x7f3e0c0d31c0 2023-12-04T05:40:51.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e12ce5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c15d890 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e12ce5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0d0e00 con 0x7f3e0c0d2880 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e10a81640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3e0c15d4b0 0x7f3e0c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e11282640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3e0c0d31c0 0x7f3e0c0b6360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36956/0 (socket says 172.21.15.111:36956) 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e10a81640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3e0c15d4b0 0x7f3e0c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43964/0 (socket says 172.21.15.111:43964) 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.297+0000 7f3e11282640 1 -- 172.21.15.111:0/2503237264 learned_addr learned my addr 172.21.15.111:0/2503237264 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e10a81640 1 -- 172.21.15.111:0/2503237264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 msgr2=0x7f3e0c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e0bfff640 1 --2- 172.21.15.111:0/2503237264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e10a81640 1 --2- 172.21.15.111:0/2503237264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e0b7fe640 1 -- 172.21.15.111:0/2503237264 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679639693 0 0) 0x7f3e0c0d2d80 con 0x7f3e0c0d31c0 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e10a81640 1 -- 172.21.15.111:0/2503237264 >> v1:172.21.15.111:6789/0 conn(0x7f3e0c0d31c0 legacy=0x7f3e0c0b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e10a81640 1 -- 172.21.15.111:0/2503237264 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0d1010 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.298+0000 7f3e10a81640 1 --2- 172.21.15.111:0/2503237264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0d1af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0097a0 tx=0x7f3dfc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc0a046f7b3fdd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e0b7fe640 1 -- 172.21.15.111:0/2503237264 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc031830 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e0b7fe640 1 -- 172.21.15.111:0/2503237264 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc031990 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e0b7fe640 1 -- 172.21.15.111:0/2503237264 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc031d20 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 -- 172.21.15.111:0/2503237264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 msgr2=0x7f3e0c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/2503237264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0d1af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0097a0 tx=0x7f3dfc02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 -- 172.21.15.111:0/2503237264 shutdown_connections 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/2503237264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/2503237264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0d1af0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 -- 172.21.15.111:0/2503237264 >> 172.21.15.111:0/2503237264 conn(0x7f3e0c072010 msgr2=0x7f3e0c072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 -- 172.21.15.111:0/2503237264 shutdown_connections 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.299+0000 7f3e12ce5640 1 -- 172.21.15.111:0/2503237264 wait complete. 2023-12-04T05:40:51.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.300+0000 7f3e12ce5640 1 Processor -- start 2023-12-04T05:40:51.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.300+0000 7f3e12ce5640 1 -- start start 2023-12-04T05:40:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e12ce5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e12ce5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e0c0d31c0 0x7f3e0c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e12ce5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e12ce5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c15f080 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e12ce5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c15e4b0 con 0x7f3e0c0d31c0 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e12ce5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c15e750 con 0x7f3e0c0d2880 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e11282640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e11282640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43970/0 (socket says 172.21.15.111:43970) 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e11282640 1 -- 172.21.15.111:0/1850489925 learned_addr learned my addr 172.21.15.111:0/1850489925 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.301+0000 7f3e10a81640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e11282640 1 -- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 msgr2=0x7f3e0c14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e11282640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c14acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e0bfff640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e0c0d31c0 0x7f3e0c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e11282640 1 -- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e0c0d31c0 msgr2=0x7f3e0c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e11282640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e0c0d31c0 0x7f3e0c0cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e11282640 1 -- 172.21.15.111:0/1850489925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc009400 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e11282640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0b75d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c7e0 tx=0x7f3e0400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400a640 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0400a7c0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c15ea30 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400ab70 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.302+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c0ccee0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.303+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c0693d0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.303+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3e0400acd0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.304+0000 7f3e097fa640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3df806b2a0 0x7f3df806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.304+0000 7f3e10a81640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3df806b2a0 0x7f3df806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.304+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e04089460 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.304+0000 7f3e10a81640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3df806b2a0 0x7f3df806d760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc036000 tx=0x7f3dfc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.308+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e040542d0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.517+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e0c15f1c0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.517+0000 7f3e097fa640 1 -- 172.21.15.111:0/1850489925 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f3e040578f0 con 0x7f3e0c15d4b0 2023-12-04T05:40:51.519 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:51.519 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:40:10.723946+0000","modified":"2023-12-04T05:40:13.542461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/975894294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":975894294},{"type":"v1","addr":"172.21.15.111:6837","nonce":975894294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-04T05:40:51.519 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 14 2023-12-04T05:40:51.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3df806b2a0 msgr2=0x7f3df806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3df806b2a0 0x7f3df806d760 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc036000 tx=0x7f3dfc008030 comp rx=0 tx=0).stop 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 msgr2=0x7f3e0c0b75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0b75d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c7e0 tx=0x7f3e0400cca0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 shutdown_connections 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3df806b2a0 0x7f3df806d760 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e0c15d4b0 0x7f3e0c0b75d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e0c0d31c0 0x7f3e0c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 --2- 172.21.15.111:0/1850489925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e0c0d2880 0x7f3e0c14acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 >> 172.21.15.111:0/1850489925 conn(0x7f3e0c072010 msgr2=0x7f3e0c15b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 shutdown_connections 2023-12-04T05:40:51.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.519+0000 7f3e12ce5640 1 -- 172.21.15.111:0/1850489925 wait complete. 2023-12-04T05:40:51.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:51.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.629+0000 7f927bb4c640 1 Processor -- start 2023-12-04T05:40:51.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.629+0000 7f927bb4c640 1 -- start start 2023-12-04T05:40:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f927bb4c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92741544c0 0x7f927406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f927bb4c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f927406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f927bb4c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927406df90 con 0x7f927406ba10 2023-12-04T05:40:51.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f927bb4c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927406bec0 con 0x7f92741544c0 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f927bb4c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740ad5d0 con 0x7f927406d250 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f92798e8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92741544c0 0x7f927406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f92798e8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f92741544c0 0x7f927406cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43976/0 (socket says 172.21.15.111:43976) 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f92798e8640 1 -- 172.21.15.111:0/1645375055 learned_addr learned my addr 172.21.15.111:0/1645375055 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f927a0e9640 1 --1- 172.21.15.111:0/1645375055 >> v1:172.21.15.111:6789/0 conn(0x7f927406ba10 0x7f9274160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36960/0 (socket says 172.21.15.111:36960) 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.630+0000 7f92790e7640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f927406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92798e8640 1 -- 172.21.15.111:0/1645375055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 msgr2=0x7f927406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92798e8640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f927406b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92798e8640 1 -- 172.21.15.111:0/1645375055 >> v1:172.21.15.111:6789/0 conn(0x7f927406ba10 legacy=0x7f9274160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92798e8640 1 -- 172.21.15.111:0/1645375055 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92741548a0 con 0x7f92741544c0 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92788e6640 1 -- 172.21.15.111:0/1645375055 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134304122 0 0) 0x7f927406df90 con 0x7f927406ba10 2023-12-04T05:40:51.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92790e7640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f927406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92798e8640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927406cd10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9270009e50 tx=0x7f927002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e7e4895cfcd4a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92788e6640 1 -- 172.21.15.111:0/1645375055 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927002cb00 con 0x7f92741544c0 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92788e6640 1 -- 172.21.15.111:0/1645375055 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927002cc60 con 0x7f92741544c0 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.631+0000 7f92788e6640 1 -- 172.21.15.111:0/1645375055 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700343f0 con 0x7f92741544c0 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 -- 172.21.15.111:0/1645375055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 msgr2=0x7f927406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927406cd10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9270009e50 tx=0x7f927002bf60 comp rx=0 tx=0).stop 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 -- 172.21.15.111:0/1645375055 shutdown_connections 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f927406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 --2- 172.21.15.111:0/1645375055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927406cd10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 -- 172.21.15.111:0/1645375055 >> 172.21.15.111:0/1645375055 conn(0x7f92740b1b50 msgr2=0x7f9274070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:51.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 -- 172.21.15.111:0/1645375055 shutdown_connections 2023-12-04T05:40:51.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.632+0000 7f927bb4c640 1 -- 172.21.15.111:0/1645375055 wait complete. 2023-12-04T05:40:51.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.633+0000 7f927bb4c640 1 Processor -- start 2023-12-04T05:40:51.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.633+0000 7f927bb4c640 1 -- start start 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927bb4c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406ba10 0x7f927414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927bb4c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f92740b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927bb4c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927bb4c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92741f2c70 con 0x7f92741544c0 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927bb4c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92741f2e80 con 0x7f927406ba10 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927bb4c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92741f3160 con 0x7f927406d250 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43978/0 (socket says 172.21.15.111:43978) 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 -- 172.21.15.111:0/2643263944 learned_addr learned my addr 172.21.15.111:0/2643263944 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:51.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f92798e8640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406ba10 0x7f927414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f92790e7640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f92740b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 -- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 msgr2=0x7f92740b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f92740b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 -- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406ba10 msgr2=0x7f927414e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406ba10 0x7f927414e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 -- 172.21.15.111:0/2643263944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270009b30 con 0x7f92741544c0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f92798e8640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406ba10 0x7f927414e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f92790e7640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f92740b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.634+0000 7f927a0e9640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927415f460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c450 tx=0x7f926c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.635+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c0074e0 con 0x7f92741544c0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.635+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c007660 con 0x7f92741544c0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.635+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c004a20 con 0x7f92741544c0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.635+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92741f3440 con 0x7f92741544c0 2023-12-04T05:40:51.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.635+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92741f3a00 con 0x7f92741544c0 2023-12-04T05:40:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.636+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92741f3c10 con 0x7f92741544c0 2023-12-04T05:40:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.636+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f926c004b80 con 0x7f92741544c0 2023-12-04T05:40:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.637+0000 7f926a7fc640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925806b2a0 0x7f925806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.637+0000 7f92798e8640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925806b2a0 0x7f925806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.637+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f926c088ce0 con 0x7f92741544c0 2023-12-04T05:40:51.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.637+0000 7f92798e8640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925806b2a0 0x7f925806d760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9270035000 tx=0x7f9270030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.641+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f926c053ca0 con 0x7f92741544c0 2023-12-04T05:40:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.809+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92740693d0 con 0x7f92741544c0 2023-12-04T05:40:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.810+0000 7f926a7fc640 1 -- 172.21.15.111:0/2643263944 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f926c018020 con 0x7f92741544c0 2023-12-04T05:40:51.811 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:51.812 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":33,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:12.395705+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:40:09.160281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-04T05:40:09.699085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925806b2a0 msgr2=0x7f925806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925806b2a0 0x7f925806d760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9270035000 tx=0x7f9270030000 comp rx=0 tx=0).stop 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 msgr2=0x7f927415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927415f460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c450 tx=0x7f926c00c910 comp rx=0 tx=0).stop 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 shutdown_connections 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f925806b2a0 0x7f925806d760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92741544c0 0x7f927415f460 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f927406d250 0x7f92740b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 --2- 172.21.15.111:0/2643263944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f927406ba10 0x7f927414e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 >> 172.21.15.111:0/2643263944 conn(0x7f92740b1b50 msgr2=0x7f92740ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 shutdown_connections 2023-12-04T05:40:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.812+0000 7f927bb4c640 1 -- 172.21.15.111:0/2643263944 wait complete. 2023-12-04T05:40:51.824 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-04T05:40:51.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.924+0000 7fd924f97640 1 Processor -- start 2023-12-04T05:40:51.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.924+0000 7fd924f97640 1 -- start start 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd924f97640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9200d1780 0x7fd9200d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd924f97640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd9200b2ad0 0x7fd9200b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd924f97640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9200cdcb0 con 0x7fd9200b20a0 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd924f97640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200d39e0 con 0x7fd9200d1780 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd924f97640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200d3cc0 con 0x7fd9200b2ad0 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd91e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9200d1780 0x7fd9200d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd91dd9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd9200b20a0 0x7fd9200d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36970/0 (socket says 172.21.15.111:36970) 2023-12-04T05:40:51.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.925+0000 7fd91dd9b640 1 -- 172.21.15.111:0/1016207239 learned_addr learned my addr 172.21.15.111:0/1016207239 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:51.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91ed9d640 1 --2- 172.21.15.111:0/1016207239 >> v2:172.21.15.192:3300/0 conn(0x7fd9200b2ad0 0x7fd9200b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91e59c640 1 -- 172.21.15.111:0/1016207239 >> v2:172.21.15.192:3300/0 conn(0x7fd9200b2ad0 msgr2=0x7fd9200b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:51.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91e59c640 1 --2- 172.21.15.111:0/1016207239 >> v2:172.21.15.192:3300/0 conn(0x7fd9200b2ad0 0x7fd9200b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91e59c640 1 -- 172.21.15.111:0/1016207239 >> v1:172.21.15.111:6789/0 conn(0x7fd9200b20a0 legacy=0x7fd9200d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91e59c640 1 -- 172.21.15.111:0/1016207239 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200cdf50 con 0x7fd9200d1780 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91e59c640 1 --2- 172.21.15.111:0/1016207239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200d1780 0x7fd9200d01e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd92008d320 tx=0x7fd91002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fde7bfeefc5b3d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91d59a640 1 -- 172.21.15.111:0/1016207239 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91002cbf0 con 0x7fd9200d1780 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91d59a640 1 -- 172.21.15.111:0/1016207239 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91002cd50 con 0x7fd9200d1780 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.926+0000 7fd91d59a640 1 -- 172.21.15.111:0/1016207239 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910034660 con 0x7fd9200d1780 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 -- 172.21.15.111:0/1016207239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200d1780 msgr2=0x7fd9200d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 --2- 172.21.15.111:0/1016207239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200d1780 0x7fd9200d01e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd92008d320 tx=0x7fd91002be40 comp rx=0 tx=0).stop 2023-12-04T05:40:51.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 -- 172.21.15.111:0/1016207239 shutdown_connections 2023-12-04T05:40:51.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 --2- 172.21.15.111:0/1016207239 >> v2:172.21.15.192:3300/0 conn(0x7fd9200b2ad0 0x7fd9200b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 --2- 172.21.15.111:0/1016207239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200d1780 0x7fd9200d01e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 -- 172.21.15.111:0/1016207239 >> 172.21.15.111:0/1016207239 conn(0x7fd92009c4e0 msgr2=0x7fd9200d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:51.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 -- 172.21.15.111:0/1016207239 shutdown_connections 2023-12-04T05:40:51.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.927+0000 7fd924f97640 1 -- 172.21.15.111:0/1016207239 wait complete. 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.928+0000 7fd924f97640 1 Processor -- start 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.928+0000 7fd924f97640 1 -- start start 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd924f97640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9200b20a0 0x7fd9201f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd924f97640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 0x7fd9200cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd924f97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9200d1780 0x7fd9200cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd924f97640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920161da0 con 0x7fd9200b2ad0 2023-12-04T05:40:51.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd924f97640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920161120 con 0x7fd9200d1780 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd924f97640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd920161400 con 0x7fd9200b20a0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 0x7fd9200cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 0x7fd9200cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44002/0 (socket says 172.21.15.111:44002) 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd91dd9b640 1 -- 172.21.15.111:0/4096196861 learned_addr learned my addr 172.21.15.111:0/4096196861 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd91ed9d640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9200d1780 0x7fd9200cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.929+0000 7fd91e59c640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9200b20a0 0x7fd9201f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91dd9b640 1 -- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9200b20a0 msgr2=0x7fd9201f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91dd9b640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9200b20a0 0x7fd9201f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91dd9b640 1 -- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9200d1780 msgr2=0x7fd9200cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91dd9b640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9200d1780 0x7fd9200cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91dd9b640 1 -- 172.21.15.111:0/4096196861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009a70 con 0x7fd9200b2ad0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91ed9d640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9200d1780 0x7fd9200cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd91dd9b640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 0x7fd9200cbc90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd91400c4a0 tx=0x7fd91400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914005ce0 con 0x7fd9200b2ad0 2023-12-04T05:40:51.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd914005e40 con 0x7fd9200b2ad0 2023-12-04T05:40:51.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9201616e0 con 0x7fd9200b2ad0 2023-12-04T05:40:51.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914007270 con 0x7fd9200b2ad0 2023-12-04T05:40:51.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.930+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920160980 con 0x7fd9200b2ad0 2023-12-04T05:40:51.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.931+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd920161ee0 con 0x7fd9200b2ad0 2023-12-04T05:40:51.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.931+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd914007490 con 0x7fd9200b2ad0 2023-12-04T05:40:51.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.931+0000 7fd90effd640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd90006b2a0 0x7fd90006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:51.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.932+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd914089090 con 0x7fd9200b2ad0 2023-12-04T05:40:51.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.932+0000 7fd91e59c640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd90006b2a0 0x7fd90006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:51.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.932+0000 7fd91e59c640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd90006b2a0 0x7fd90006d760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd910036000 tx=0x7fd910030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:51.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:51.936+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd920161ee0 con 0x7fd9200b2ad0 2023-12-04T05:40:52.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.138+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd920161ee0 con 0x7fd9200b2ad0 2023-12-04T05:40:52.140 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-12-04T05:40:52.139+0000 7fb27d104640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-04T05:40:52.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.140+0000 7fd90effd640 1 -- 172.21.15.111:0/4096196861 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd920161ee0 con 0x7fd9200b2ad0 2023-12-04T05:40:52.141 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-04T05:40:52.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd90006b2a0 msgr2=0x7fd90006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:52.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd90006b2a0 0x7fd90006d760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd910036000 tx=0x7fd910030000 comp rx=0 tx=0).stop 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 msgr2=0x7fd9200cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 0x7fd9200cbc90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd91400c4a0 tx=0x7fd91400c960 comp rx=0 tx=0).stop 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 shutdown_connections 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd90006b2a0 0x7fd90006d760 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9200d1780 0x7fd9200cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9200b2ad0 0x7fd9200cbc90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 --2- 172.21.15.111:0/4096196861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9200b20a0 0x7fd9201f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 >> 172.21.15.111:0/4096196861 conn(0x7fd92009c4e0 msgr2=0x7fd9200b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 shutdown_connections 2023-12-04T05:40:52.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.142+0000 7fd924f97640 1 -- 172.21.15.111:0/4096196861 wait complete. 2023-12-04T05:40:52.146 INFO:tasks.ceph.mds.c.smithi111.stderr: -1703> 2023-12-04T05:40:15.366+0000 7f32d6815640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-04T05:40:52.156 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-04T05:40:52.170 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2023-12-04T05:40:52.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.259+0000 7f6f60db0640 1 Processor -- start 2023-12-04T05:40:52.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.259+0000 7f6f60db0640 1 -- start start 2023-12-04T05:40:52.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f60db0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f5c0d3140 0x7f6f5c0d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f60db0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c15d5a0 0x7f6f5c0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f60db0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f5c0d2800 0x7f6f5c0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f60db0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c0d3a60 con 0x7f6f5c0d2800 2023-12-04T05:40:52.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f60db0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c0d2d80 con 0x7f6f5c0d3140 2023-12-04T05:40:52.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f60db0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c0d0d80 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f5ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f5c0d2800 0x7f6f5c0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f5ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6f5c0d2800 0x7f6f5c0ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44026/0 (socket says 172.21.15.111:44026) 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.260+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3600485681 learned_addr learned my addr 172.21.15.111:0/3600485681 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3600485681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c15d5a0 msgr2=0x7f6f5c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5ad9d640 1 --2- 172.21.15.111:0/3600485681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c15d5a0 0x7f6f5c0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3600485681 >> v2:172.21.15.192:3300/0 conn(0x7f6f5c0d3140 msgr2=0x7f6f5c0d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5a59c640 1 --2- 172.21.15.111:0/3600485681 >> v2:172.21.15.192:3300/0 conn(0x7f6f5c0d3140 0x7f6f5c0d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5ad9d640 1 --2- 172.21.15.111:0/3600485681 >> v2:172.21.15.192:3300/0 conn(0x7f6f5c0d3140 0x7f6f5c0d3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3600485681 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0d1060 con 0x7f6f5c0d2800 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5ad9d640 1 --2- 172.21.15.111:0/3600485681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c0d2800 0x7f6f5c0ce230 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0cfb90 tx=0x7f6f5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578f87ed420d3391 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5959a640 1 -- 172.21.15.111:0/3600485681 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54031870 con 0x7f6f5c0d2800 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5959a640 1 -- 172.21.15.111:0/3600485681 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f540319d0 con 0x7f6f5c0d2800 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.261+0000 7f6f5959a640 1 -- 172.21.15.111:0/3600485681 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54031d60 con 0x7f6f5c0d2800 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 -- 172.21.15.111:0/3600485681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c0d2800 msgr2=0x7f6f5c0ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3600485681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c0d2800 0x7f6f5c0ce230 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0cfb90 tx=0x7f6f5402b910 comp rx=0 tx=0).stop 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 -- 172.21.15.111:0/3600485681 shutdown_connections 2023-12-04T05:40:52.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3600485681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c0d2800 0x7f6f5c0ce230 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3600485681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c15d5a0 0x7f6f5c0d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3600485681 >> v2:172.21.15.192:3300/0 conn(0x7f6f5c0d3140 0x7f6f5c0d3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 -- 172.21.15.111:0/3600485681 >> 172.21.15.111:0/3600485681 conn(0x7f6f5c072010 msgr2=0x7f6f5c072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 -- 172.21.15.111:0/3600485681 shutdown_connections 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.262+0000 7f6f60db0640 1 -- 172.21.15.111:0/3600485681 wait complete. 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.263+0000 7f6f60db0640 1 Processor -- start 2023-12-04T05:40:52.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.263+0000 7f6f60db0640 1 -- start start 2023-12-04T05:40:52.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f60db0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c0d2800 0x7f6f5c150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f60db0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f5c0d3140 0x7f6f5c15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f60db0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 0x7f6f5c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f60db0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c160520 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f60db0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1554e0 con 0x7f6f5c0d3140 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f60db0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c15f9c0 con 0x7f6f5c0d2800 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 0x7f6f5c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 0x7f6f5c15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44034/0 (socket says 172.21.15.111:44034) 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3782516222 learned_addr learned my addr 172.21.15.111:0/3782516222 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5a59c640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c0d2800 0x7f6f5c150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f59d9b640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f5c0d3140 0x7f6f5c15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c0d2800 msgr2=0x7f6f5c150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:52.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c0d2800 0x7f6f5c150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f5c0d3140 msgr2=0x7f6f5c15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f5c0d3140 0x7f6f5c15e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.264+0000 7f6f5ad9d640 1 -- 172.21.15.111:0/3782516222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540094d0 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.265+0000 7f6f5ad9d640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 0x7f6f5c15eaf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6f540090b0 tx=0x7f6f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.265+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5403adf0 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.265+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f540443f0 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.265+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c15fc80 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.265+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540447a0 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.265+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c160240 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.266+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c155300 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.266+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6f5404d450 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.266+0000 7f6f4affd640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3c06b2a0 0x7f6f3c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:52.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.267+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6f5402d040 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.267+0000 7f6f5a59c640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3c06b2a0 0x7f6f3c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:52.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.267+0000 7f6f5a59c640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3c06b2a0 0x7f6f3c06d760 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0c6200 tx=0x7f6f4c008460 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:52.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.271+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f54077070 con 0x7f6f5c15d5a0 2023-12-04T05:40:52.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:52.472+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f5c0d2000 con 0x7f6f5c15d5a0 2023-12-04T05:40:53.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.141+0000 7f6f4affd640 1 -- 172.21.15.111:0/3782516222 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6f5407a690 con 0x7f6f5c15d5a0 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3c06b2a0 msgr2=0x7f6f3c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3c06b2a0 0x7f6f3c06d760 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0c6200 tx=0x7f6f4c008460 comp rx=0 tx=0).stop 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 msgr2=0x7f6f5c15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 0x7f6f5c15eaf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6f540090b0 tx=0x7f6f540046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 shutdown_connections 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6f3c06b2a0 0x7f6f3c06d760 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c15d5a0 0x7f6f5c15eaf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f5c0d3140 0x7f6f5c15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 --2- 172.21.15.111:0/3782516222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f5c0d2800 0x7f6f5c150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 >> 172.21.15.111:0/3782516222 conn(0x7f6f5c072010 msgr2=0x7f6f5c0b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:53.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 shutdown_connections 2023-12-04T05:40:53.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.145+0000 7f6f60db0640 1 -- 172.21.15.111:0/3782516222 wait complete. 2023-12-04T05:40:53.159 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-04T05:40:53.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.306+0000 7fbe4ece7640 1 Processor -- start 2023-12-04T05:40:53.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.306+0000 7fbe4ece7640 1 -- start start 2023-12-04T05:40:53.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ece7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbe4815d560 0x7fbe480b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:53.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ece7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbe480d11b0 0x7fbe480b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:53.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ece7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe480d27f0 con 0x7fbe480b5110 2023-12-04T05:40:53.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ece7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe480b5580 con 0x7fbe4815d560 2023-12-04T05:40:53.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ece7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe480d1750 con 0x7fbe480d11b0 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ca83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbe4815d560 0x7fbe480b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ca83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbe4815d560 0x7fbe480b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44042/0 (socket says 172.21.15.111:44042) 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4ca83640 1 -- 172.21.15.111:0/3822858041 learned_addr learned my addr 172.21.15.111:0/3822858041 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.307+0000 7fbe4d284640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbe480b5110 0x7fbe4806b540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36978/0 (socket says 172.21.15.111:36978) 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe47fff640 1 --2- 172.21.15.111:0/3822858041 >> v2:172.21.15.192:3300/0 conn(0x7fbe480d11b0 0x7fbe480b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe4ca83640 1 -- 172.21.15.111:0/3822858041 >> v2:172.21.15.192:3300/0 conn(0x7fbe480d11b0 msgr2=0x7fbe480b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe4ca83640 1 --2- 172.21.15.111:0/3822858041 >> v2:172.21.15.192:3300/0 conn(0x7fbe480d11b0 0x7fbe480b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe4ca83640 1 -- 172.21.15.111:0/3822858041 >> v1:172.21.15.111:6789/0 conn(0x7fbe480b5110 legacy=0x7fbe4806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe4ca83640 1 -- 172.21.15.111:0/3822858041 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe480d1a30 con 0x7fbe4815d560 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe477fe640 1 -- 172.21.15.111:0/3822858041 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732011242 0 0) 0x7fbe480d27f0 con 0x7fbe480b5110 2023-12-04T05:40:53.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe47fff640 1 --2- 172.21.15.111:0/3822858041 >> v2:172.21.15.192:3300/0 conn(0x7fbe480d11b0 0x7fbe480b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe4ca83640 1 --2- 172.21.15.111:0/3822858041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe4815d560 0x7fbe480b8b20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbe480d0c00 tx=0x7fbe3802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78259f51fba1f6d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe477fe640 1 -- 172.21.15.111:0/3822858041 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38030860 con 0x7fbe4815d560 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe477fe640 1 -- 172.21.15.111:0/3822858041 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe380309c0 con 0x7fbe4815d560 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.308+0000 7fbe477fe640 1 -- 172.21.15.111:0/3822858041 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38030d50 con 0x7fbe4815d560 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 -- 172.21.15.111:0/3822858041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe4815d560 msgr2=0x7fbe480b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/3822858041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe4815d560 0x7fbe480b8b20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbe480d0c00 tx=0x7fbe3802b9a0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 -- 172.21.15.111:0/3822858041 shutdown_connections 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/3822858041 >> v2:172.21.15.192:3300/0 conn(0x7fbe480d11b0 0x7fbe480b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/3822858041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe4815d560 0x7fbe480b8b20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 -- 172.21.15.111:0/3822858041 >> 172.21.15.111:0/3822858041 conn(0x7fbe48072220 msgr2=0x7fbe480cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 -- 172.21.15.111:0/3822858041 shutdown_connections 2023-12-04T05:40:53.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.309+0000 7fbe4ece7640 1 -- 172.21.15.111:0/3822858041 wait complete. 2023-12-04T05:40:53.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.310+0000 7fbe4ece7640 1 Processor -- start 2023-12-04T05:40:53.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.310+0000 7fbe4ece7640 1 -- start start 2023-12-04T05:40:53.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ece7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 0x7fbe48160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:53.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ece7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480d11b0 0x7fbe480c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:53.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ece7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe4814f180 0x7fbe480b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ece7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4814f620 con 0x7fbe480b5110 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ece7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4814f900 con 0x7fbe480d11b0 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ece7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4814fbe0 con 0x7fbe4814f180 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 0x7fbe48160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 0x7fbe48160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44058/0 (socket says 172.21.15.111:44058) 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 -- 172.21.15.111:0/277584367 learned_addr learned my addr 172.21.15.111:0/277584367 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe47fff640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480d11b0 0x7fbe480c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 -- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe4814f180 msgr2=0x7fbe480b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe4814f180 0x7fbe480b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 -- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480d11b0 msgr2=0x7fbe480c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480d11b0 0x7fbe480c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.311+0000 7fbe4ca83640 1 -- 172.21.15.111:0/277584367 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380094d0 con 0x7fbe480b5110 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.312+0000 7fbe4ca83640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 0x7fbe48160230 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe3802b9e0 tx=0x7fbe38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.312+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38030850 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.312+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38039d90 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.312+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4814fec0 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.312+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38043650 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.312+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe48160770 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.313+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe480d2290 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.313+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbe3804c3b0 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.313+0000 7fbe457fa640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbe3406b2a0 0x7fbe3406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.314+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbe380ac3d0 con 0x7fbe480b5110 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.314+0000 7fbe47fff640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbe3406b2a0 0x7fbe3406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.314+0000 7fbe47fff640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbe3406b2a0 0x7fbe3406d760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe4815c370 tx=0x7fbe3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:53.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.318+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe380771e0 con 0x7fbe480b5110 2023-12-04T05:40:53.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:53.485+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe481503e0 con 0x7fbe480b5110 2023-12-04T05:40:54.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.259+0000 7fbe457fa640 1 -- 172.21.15.111:0/277584367 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbe3807a800 con 0x7fbe480b5110 2023-12-04T05:40:54.261 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbe3406b2a0 msgr2=0x7fbe3406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbe3406b2a0 0x7fbe3406d760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe4815c370 tx=0x7fbe3c008040 comp rx=0 tx=0).stop 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 msgr2=0x7fbe48160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 0x7fbe48160230 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe3802b9e0 tx=0x7fbe38004730 comp rx=0 tx=0).stop 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 shutdown_connections 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbe3406b2a0 0x7fbe3406d760 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe4814f180 0x7fbe480b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480d11b0 0x7fbe480c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 --2- 172.21.15.111:0/277584367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe480b5110 0x7fbe48160230 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.262+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 >> 172.21.15.111:0/277584367 conn(0x7fbe48072220 msgr2=0x7fbe480b6230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:54.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.263+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 shutdown_connections 2023-12-04T05:40:54.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.263+0000 7fbe4ece7640 1 -- 172.21.15.111:0/277584367 wait complete. 2023-12-04T05:40:54.276 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-04T05:40:54.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.385+0000 7fa1cab08640 1 Processor -- start 2023-12-04T05:40:54.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.385+0000 7fa1cab08640 1 -- start start 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1cab08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa1c406c570 0x7fa1c40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1cab08640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c415fe10 con 0x7fa1c415ef90 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1cab08640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c40ce670 con 0x7fa1c40ce090 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1cab08640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c406c980 con 0x7fa1c406c570 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c90a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa1c406c570 0x7fa1c40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c88a4640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa1c415ef90 0x7fa1c415f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36982/0 (socket says 172.21.15.111:36982) 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c90a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa1c406c570 0x7fa1c40cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44060/0 (socket says 172.21.15.111:44060) 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c88a4640 1 -- 172.21.15.111:0/1385854068 learned_addr learned my addr 172.21.15.111:0/1385854068 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:54.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c90a5640 1 -- 172.21.15.111:0/1385854068 >> v1:172.21.15.192:6789/0 conn(0x7fa1c40ce090 legacy=0x7fa1c40cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:54.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c90a5640 1 -- 172.21.15.111:0/1385854068 >> v1:172.21.15.111:6789/0 conn(0x7fa1c415ef90 legacy=0x7fa1c415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:54.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c90a5640 1 -- 172.21.15.111:0/1385854068 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c40caa80 con 0x7fa1c406c570 2023-12-04T05:40:54.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.386+0000 7fa1c37fe640 1 -- 172.21.15.111:0/1385854068 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113242220 0 0) 0x7fa1c415fe10 con 0x7fa1c415ef90 2023-12-04T05:40:54.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.387+0000 7fa1c90a5640 1 --2- 172.21.15.111:0/1385854068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c406c570 0x7fa1c40cd0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009d90 tx=0x7fa1bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386291d3bd1e606b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:54.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.387+0000 7fa1c37fe640 1 -- 172.21.15.111:0/1385854068 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc02cc60 con 0x7fa1c406c570 2023-12-04T05:40:54.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.387+0000 7fa1c37fe640 1 -- 172.21.15.111:0/1385854068 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc02cdc0 con 0x7fa1c406c570 2023-12-04T05:40:54.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.387+0000 7fa1c37fe640 1 -- 172.21.15.111:0/1385854068 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc034660 con 0x7fa1c406c570 2023-12-04T05:40:54.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 -- 172.21.15.111:0/1385854068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c406c570 msgr2=0x7fa1c40cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:54.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 --2- 172.21.15.111:0/1385854068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c406c570 0x7fa1c40cd0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009d90 tx=0x7fa1bc02beb0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 -- 172.21.15.111:0/1385854068 shutdown_connections 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 --2- 172.21.15.111:0/1385854068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c406c570 0x7fa1c40cd0a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 -- 172.21.15.111:0/1385854068 >> 172.21.15.111:0/1385854068 conn(0x7fa1c40b0c80 msgr2=0x7fa1c40b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 -- 172.21.15.111:0/1385854068 shutdown_connections 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.388+0000 7fa1cab08640 1 -- 172.21.15.111:0/1385854068 wait complete. 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.389+0000 7fa1cab08640 1 Processor -- start 2023-12-04T05:40:54.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.389+0000 7fa1cab08640 1 -- start start 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1cab08640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1c406c570 0x7fa1c409a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1cab08640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c40ce090 0x7fa1c414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1cab08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 0x7fa1c40c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1cab08640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4152d60 con 0x7fa1c415ef90 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1cab08640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4152a80 con 0x7fa1c40ce090 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1cab08640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4099b10 con 0x7fa1c406c570 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 0x7fa1c40c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 0x7fa1c40c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44064/0 (socket says 172.21.15.111:44064) 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 -- 172.21.15.111:0/2217933040 learned_addr learned my addr 172.21.15.111:0/2217933040 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c88a4640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1c406c570 0x7fa1c409a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:54.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c3fff640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c40ce090 0x7fa1c414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 -- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1c406c570 msgr2=0x7fa1c409a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1c406c570 0x7fa1c409a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 -- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c40ce090 msgr2=0x7fa1c414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c40ce090 0x7fa1c414bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c90a5640 1 -- 172.21.15.111:0/2217933040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc009a70 con 0x7fa1c415ef90 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c3fff640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c40ce090 0x7fa1c414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.390+0000 7fa1c88a4640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1c406c570 0x7fa1c409a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.391+0000 7fa1c90a5640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 0x7fa1c40c8300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009d90 tx=0x7fa1bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.391+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc03e610 con 0x7fa1c415ef90 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.391+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc03e770 con 0x7fa1c415ef90 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.391+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c4099df0 con 0x7fa1c415ef90 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.391+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc03eb60 con 0x7fa1c415ef90 2023-12-04T05:40:54.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.391+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c409ab30 con 0x7fa1c415ef90 2023-12-04T05:40:54.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.392+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c409a2b0 con 0x7fa1c415ef90 2023-12-04T05:40:54.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.392+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa1bc03b050 con 0x7fa1c415ef90 2023-12-04T05:40:54.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.393+0000 7fa1c17fa640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa1b006b2d0 0x7fa1b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:54.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.393+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fa1bc0ac110 con 0x7fa1c415ef90 2023-12-04T05:40:54.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.393+0000 7fa1c88a4640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa1b006b2d0 0x7fa1b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:54.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.393+0000 7fa1c88a4640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa1b006b2d0 0x7fa1b006d790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4000f60 tx=0x7fa1b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:54.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.397+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1bc0771a0 con 0x7fa1c415ef90 2023-12-04T05:40:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:54.567+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1c4069360 con 0x7fa1c415ef90 2023-12-04T05:40:55.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.264+0000 7fa1c17fa640 1 -- 172.21.15.111:0/2217933040 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa1bc07a7c0 con 0x7fa1c415ef90 2023-12-04T05:40:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2023-12-04T05:40:55.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.267+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa1b006b2d0 msgr2=0x7fa1b006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.267+0000 7fa1cab08640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa1b006b2d0 0x7fa1b006d790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4000f60 tx=0x7fa1b4009040 comp rx=0 tx=0).stop 2023-12-04T05:40:55.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.267+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 msgr2=0x7fa1c40c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.267+0000 7fa1cab08640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 0x7fa1c40c8300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009d90 tx=0x7fa1bc004a40 comp rx=0 tx=0).stop 2023-12-04T05:40:55.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 shutdown_connections 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa1b006b2d0 0x7fa1b006d790 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1c415ef90 0x7fa1c40c8300 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c40ce090 0x7fa1c414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 --2- 172.21.15.111:0/2217933040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1c406c570 0x7fa1c409a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 >> 172.21.15.111:0/2217933040 conn(0x7fa1c40b0c80 msgr2=0x7fa1c40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 shutdown_connections 2023-12-04T05:40:55.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.268+0000 7fa1cab08640 1 -- 172.21.15.111:0/2217933040 wait complete. 2023-12-04T05:40:55.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2023-12-04T05:40:55.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.385+0000 7f52cdcda640 1 Processor -- start 2023-12-04T05:40:55.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.385+0000 7f52cdcda640 1 -- start start 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52cdcda640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c8150680 0x7f52c8071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52cdcda640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c815a3f0 0x7f52c806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52cdcda640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c815a9d0 con 0x7f52c806cac0 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52cdcda640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c806d4b0 con 0x7f52c8150680 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52cdcda640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c8072390 con 0x7f52c815a3f0 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c8150680 0x7f52c8071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c6ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f52c806cac0 0x7f52c8159ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36984/0 (socket says 172.21.15.111:36984) 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c8150680 0x7f52c8071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44076/0 (socket says 172.21.15.111:44076) 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c6ffd640 1 -- 172.21.15.111:0/2588928488 learned_addr learned my addr 172.21.15.111:0/2588928488 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c77fe640 1 -- 172.21.15.111:0/2588928488 >> v2:172.21.15.192:3300/0 conn(0x7f52c815a3f0 msgr2=0x7f52c806cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c77fe640 1 --2- 172.21.15.111:0/2588928488 >> v2:172.21.15.192:3300/0 conn(0x7f52c815a3f0 0x7f52c806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c77fe640 1 -- 172.21.15.111:0/2588928488 >> v1:172.21.15.111:6789/0 conn(0x7f52c806cac0 legacy=0x7f52c8159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c77fe640 1 -- 172.21.15.111:0/2588928488 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8070570 con 0x7f52c8150680 2023-12-04T05:40:55.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.386+0000 7f52c67fc640 1 -- 172.21.15.111:0/2588928488 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013094013 0 0) 0x7f52c815a9d0 con 0x7f52c806cac0 2023-12-04T05:40:55.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.387+0000 7f52c77fe640 1 --2- 172.21.15.111:0/2588928488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8071050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f52c80cc5d0 tx=0x7f52b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d7ce3d46ec26f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.387+0000 7f52c67fc640 1 -- 172.21.15.111:0/2588928488 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8031830 con 0x7f52c8150680 2023-12-04T05:40:55.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.387+0000 7f52c67fc640 1 -- 172.21.15.111:0/2588928488 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8031990 con 0x7f52c8150680 2023-12-04T05:40:55.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.387+0000 7f52c67fc640 1 -- 172.21.15.111:0/2588928488 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8031d20 con 0x7f52c8150680 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 -- 172.21.15.111:0/2588928488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 msgr2=0x7f52c8071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 --2- 172.21.15.111:0/2588928488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8071050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f52c80cc5d0 tx=0x7f52b802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 -- 172.21.15.111:0/2588928488 shutdown_connections 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 --2- 172.21.15.111:0/2588928488 >> v2:172.21.15.192:3300/0 conn(0x7f52c815a3f0 0x7f52c806cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 --2- 172.21.15.111:0/2588928488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8071050 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 -- 172.21.15.111:0/2588928488 >> 172.21.15.111:0/2588928488 conn(0x7f52c806a4c0 msgr2=0x7f52c806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 -- 172.21.15.111:0/2588928488 shutdown_connections 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.388+0000 7f52cdcda640 1 -- 172.21.15.111:0/2588928488 wait complete. 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.389+0000 7f52cdcda640 1 Processor -- start 2023-12-04T05:40:55.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.389+0000 7f52cdcda640 1 -- start start 2023-12-04T05:40:55.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52cdcda640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52c806cac0 0x7f52c80cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52cdcda640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52cdcda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c815a3f0 0x7f52c806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52cdcda640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80adb00 con 0x7f52c8150680 2023-12-04T05:40:55.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52cdcda640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80ce240 con 0x7f52c806cac0 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52cdcda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80ac600 con 0x7f52c815a3f0 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44086/0 (socket says 172.21.15.111:44086) 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 -- 172.21.15.111:0/738497774 learned_addr learned my addr 172.21.15.111:0/738497774 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 -- 172.21.15.111:0/738497774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c815a3f0 msgr2=0x7f52c806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c815a3f0 0x7f52c806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 -- 172.21.15.111:0/738497774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52c806cac0 msgr2=0x7f52c80cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52c806cac0 0x7f52c80cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.390+0000 7f52c6ffd640 1 -- 172.21.15.111:0/738497774 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8009400 con 0x7f52c8150680 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.391+0000 7f52c6ffd640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8149fb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c4d0 tx=0x7f52bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.391+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0074e0 con 0x7f52c8150680 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.391+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc007660 con 0x7f52c8150680 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.391+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c8150be0 con 0x7f52c8150680 2023-12-04T05:40:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.391+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004a20 con 0x7f52c8150680 2023-12-04T05:40:55.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.391+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c80acbc0 con 0x7f52c8150680 2023-12-04T05:40:55.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.392+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c80693d0 con 0x7f52c8150680 2023-12-04T05:40:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.392+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f52bc004b80 con 0x7f52c8150680 2023-12-04T05:40:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.393+0000 7f52affff640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f52b406b2a0 0x7f52b406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.393+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f52bc088dc0 con 0x7f52c8150680 2023-12-04T05:40:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.393+0000 7f52c77fe640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f52b406b2a0 0x7f52b406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.393+0000 7f52c77fe640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f52b406b2a0 0x7f52b406d760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52b8036000 tx=0x7f52b8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.397+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52bc053fc0 con 0x7f52c8150680 2023-12-04T05:40:55.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.568+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) v1 -- 0x7f52c80cecb0 con 0x7f52c8150680 2023-12-04T05:40:55.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.571+0000 7f52affff640 1 -- 172.21.15.111:0/738497774 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v12) v1 ==== 106+0+0 (secure 0 0 0) 0x7f52bc0180b0 con 0x7f52c8150680 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f52b406b2a0 msgr2=0x7f52b406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f52b406b2a0 0x7f52b406d760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52b8036000 tx=0x7f52b8030000 comp rx=0 tx=0).stop 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 msgr2=0x7f52c8149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8149fb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c4d0 tx=0x7f52bc00c990 comp rx=0 tx=0).stop 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 shutdown_connections 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f52b406b2a0 0x7f52b406d760 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c815a3f0 0x7f52c806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52c8150680 0x7f52c8149fb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 --2- 172.21.15.111:0/738497774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52c806cac0 0x7f52c80cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.573+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 >> 172.21.15.111:0/738497774 conn(0x7f52c806a4c0 msgr2=0x7f52c814e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:55.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.574+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 shutdown_connections 2023-12-04T05:40:55.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.574+0000 7f52cdcda640 1 -- 172.21.15.111:0/738497774 wait complete. 2023-12-04T05:40:55.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-04T05:40:55.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.685+0000 7fac9e341640 1 Processor -- start 2023-12-04T05:40:55.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.685+0000 7fac9e341640 1 -- start start 2023-12-04T05:40:55.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac9e341640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac98146a60 0x7fac980d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac9e341640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac980d16d0 0x7fac98160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac9e341640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980d1b30 con 0x7fac98161530 2023-12-04T05:40:55.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac9e341640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9815da00 con 0x7fac98146a60 2023-12-04T05:40:55.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac9e341640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98161d60 con 0x7fac980d16d0 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac98146a60 0x7fac980d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fac98146a60 0x7fac980d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44094/0 (socket says 172.21.15.111:44094) 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 -- 172.21.15.111:0/210656167 learned_addr learned my addr 172.21.15.111:0/210656167 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac977fe640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac980d16d0 0x7fac98160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 -- 172.21.15.111:0/210656167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac980d16d0 msgr2=0x7fac98160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac980d16d0 0x7fac98160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 -- 172.21.15.111:0/210656167 >> v1:172.21.15.192:6789/0 conn(0x7fac98161530 legacy=0x7fac980b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac97fff640 1 -- 172.21.15.111:0/210656167 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9815dcc0 con 0x7fac98146a60 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.686+0000 7fac977fe640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac980d16d0 0x7fac98160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac97fff640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac980d1190 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fac88009870 tx=0x7fac8802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7562572944a97f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac96ffd640 1 -- 172.21.15.111:0/210656167 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880317c0 con 0x7fac98146a60 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac96ffd640 1 -- 172.21.15.111:0/210656167 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88031920 con 0x7fac98146a60 2023-12-04T05:40:55.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac96ffd640 1 -- 172.21.15.111:0/210656167 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88031cb0 con 0x7fac98146a60 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 -- 172.21.15.111:0/210656167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 msgr2=0x7fac980d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac980d1190 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fac88009870 tx=0x7fac8802b860 comp rx=0 tx=0).stop 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 -- 172.21.15.111:0/210656167 shutdown_connections 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac980d16d0 0x7fac98160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 --2- 172.21.15.111:0/210656167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac980d1190 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 -- 172.21.15.111:0/210656167 >> 172.21.15.111:0/210656167 conn(0x7fac9806a6a0 msgr2=0x7fac9806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 -- 172.21.15.111:0/210656167 shutdown_connections 2023-12-04T05:40:55.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.687+0000 7fac9e341640 1 -- 172.21.15.111:0/210656167 wait complete. 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 Processor -- start 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 -- start start 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac980d16d0 0x7fac9813c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac9813daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac98161530 0x7fac98142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9813d610 con 0x7fac98146a60 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9813ca00 con 0x7fac980d16d0 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.689+0000 7fac9e341640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9813cce0 con 0x7fac98161530 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac9813daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac9813daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44096/0 (socket says 172.21.15.111:44096) 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 -- 172.21.15.111:0/3064878113 learned_addr learned my addr 172.21.15.111:0/3064878113 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:55.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac97fff640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac980d16d0 0x7fac9813c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac9c8de640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac98161530 0x7fac98142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 -- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac98161530 msgr2=0x7fac98142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac98161530 0x7fac98142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 -- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac980d16d0 msgr2=0x7fac9813c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac980d16d0 0x7fac9813c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 -- 172.21.15.111:0/3064878113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac880094d0 con 0x7fac98146a60 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac977fe640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac9813daa0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c7f0 tx=0x7fac8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c007a30 con 0x7fac98146a60 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c007b90 con 0x7fac98146a60 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00ce40 con 0x7fac98146a60 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9813cfc0 con 0x7fac98146a60 2023-12-04T05:40:55.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.690+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9813c660 con 0x7fac98146a60 2023-12-04T05:40:55.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.691+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac980693d0 con 0x7fac98146a60 2023-12-04T05:40:55.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.692+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fac8c0113f0 con 0x7fac98146a60 2023-12-04T05:40:55.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.692+0000 7fac94ff9640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fac8406b2a0 0x7fac8406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.692+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac8c089a80 con 0x7fac98146a60 2023-12-04T05:40:55.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.692+0000 7fac97fff640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fac8406b2a0 0x7fac8406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.692+0000 7fac97fff640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fac8406b2a0 0x7fac8406d760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac88036000 tx=0x7fac8803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.696+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac8c054c20 con 0x7fac98146a60 2023-12-04T05:40:55.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.861+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fac98070cf0 con 0x7fac98146a60 2023-12-04T05:40:55.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.865+0000 7fac94ff9640 1 -- 172.21.15.111:0/3064878113 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fac8c058240 con 0x7fac98146a60 2023-12-04T05:40:55.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.866+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fac8406b2a0 msgr2=0x7fac8406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.866+0000 7fac9e341640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fac8406b2a0 0x7fac8406d760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac88036000 tx=0x7fac8803d040 comp rx=0 tx=0).stop 2023-12-04T05:40:55.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 msgr2=0x7fac9813daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac9813daa0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c7f0 tx=0x7fac8c00cbf0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 shutdown_connections 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fac8406b2a0 0x7fac8406d760 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac98161530 0x7fac98142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac98146a60 0x7fac9813daa0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 --2- 172.21.15.111:0/3064878113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac980d16d0 0x7fac9813c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 >> 172.21.15.111:0/3064878113 conn(0x7fac9806a6a0 msgr2=0x7fac9806aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 shutdown_connections 2023-12-04T05:40:55.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.867+0000 7fac9e341640 1 -- 172.21.15.111:0/3064878113 wait complete. 2023-12-04T05:40:55.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2023-12-04T05:40:55.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.977+0000 7f3b7daee640 1 Processor -- start 2023-12-04T05:40:55.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.977+0000 7f3b7daee640 1 -- start start 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b7daee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b78144430 0x7f3b7815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b7daee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 0x7f3b78071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b7daee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b78070bd0 con 0x7f3b780721b0 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b7daee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b78144810 con 0x7f3b78144430 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b7daee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78070eb0 con 0x7f3b780703f0 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b77fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3b780721b0 0x7f3b7806c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36996/0 (socket says 172.21.15.111:36996) 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b77fff640 1 -- 172.21.15.111:0/30846055 learned_addr learned my addr 172.21.15.111:0/30846055 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b76ffd640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 0x7f3b78071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b777fe640 1 --2- 172.21.15.111:0/30846055 >> v2:172.21.15.192:3300/0 conn(0x7f3b78144430 0x7f3b7815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404845378 0 0) 0x7f3b78070bd0 con 0x7f3b780721b0 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b60003520 con 0x7f3b780721b0 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656568202 0 0) 0x7f3b60003520 con 0x7f3b780721b0 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 msgr2=0x7f3b78071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 0x7f3b78071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 >> v2:172.21.15.192:3300/0 conn(0x7f3b78144430 msgr2=0x7f3b7815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 --2- 172.21.15.111:0/30846055 >> v2:172.21.15.192:3300/0 conn(0x7f3b78144430 0x7f3b7815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78071190 con 0x7f3b780721b0 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f3b700019f0 con 0x7f3b780721b0 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 >> v1:172.21.15.111:6789/0 conn(0x7f3b780721b0 legacy=0x7f3b7806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b60003970 0x7f3b60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 0x7f3b600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60004d00 0x7f3b60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78144810 con 0x7f3b60003970 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b78070eb0 con 0x7f3b60004d00 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.978+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6000b980 con 0x7f3b780703f0 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b60003970 0x7f3b60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b76ffd640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 0x7f3b600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b77fff640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60004d00 0x7f3b60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 -- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 msgr2=0x7f3b600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b780703f0 0x7f3b600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 -- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60004d00 msgr2=0x7f3b60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60004d00 0x7f3b60005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 -- 172.21.15.111:0/30846055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b60003520 con 0x7f3b60003970 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b77fff640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60004d00 0x7f3b60005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:55.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.979+0000 7f3b777fe640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b60003970 0x7f3b60003d90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009ac0 tx=0x7f3b600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564bf09ace143cec server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c02c710 con 0x7f3b60003970 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c02c870 con 0x7f3b60003970 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b767fc640 1 -- 172.21.15.111:0/30846055 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c02cc00 con 0x7f3b60003970 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b7daee640 1 -- 172.21.15.111:0/30846055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b60003970 msgr2=0x7f3b60003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b7daee640 1 --2- 172.21.15.111:0/30846055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b60003970 0x7f3b60003d90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009ac0 tx=0x7f3b600057f0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b77fff640 1 -- 172.21.15.111:0/30846055 reap_dead start 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b7daee640 1 -- 172.21.15.111:0/30846055 shutdown_connections 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b7daee640 1 -- 172.21.15.111:0/30846055 >> 172.21.15.111:0/30846055 conn(0x7f3b7806a4c0 msgr2=0x7f3b7806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b7daee640 1 -- 172.21.15.111:0/30846055 shutdown_connections 2023-12-04T05:40:55.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.980+0000 7f3b7daee640 1 -- 172.21.15.111:0/30846055 wait complete. 2023-12-04T05:40:55.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.981+0000 7f3b7daee640 1 Processor -- start 2023-12-04T05:40:55.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 -- start start 2023-12-04T05:40:55.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60003970 0x7f3b780c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 0x7f3b781ee420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b781f2110 0x7f3b781f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780c6ec0 con 0x7f3b780a8900 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b781f2ad0 con 0x7f3b60003970 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b7daee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7815b020 con 0x7f3b781f2110 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b76ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 0x7f3b781ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b76ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 0x7f3b781ee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44120/0 (socket says 172.21.15.111:44120) 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b76ffd640 1 -- 172.21.15.111:0/1654995850 learned_addr learned my addr 172.21.15.111:0/1654995850 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b777fe640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60003970 0x7f3b780c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.982+0000 7f3b77fff640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b781f2110 0x7f3b781f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b76ffd640 1 -- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b781f2110 msgr2=0x7f3b781f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b76ffd640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b781f2110 0x7f3b781f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b76ffd640 1 -- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60003970 msgr2=0x7f3b780c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b76ffd640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60003970 0x7f3b780c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b76ffd640 1 -- 172.21.15.111:0/1654995850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78130220 con 0x7f3b780a8900 2023-12-04T05:40:55.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b777fe640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60003970 0x7f3b780c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b76ffd640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 0x7f3b781ee420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3b680119a0 tx=0x7f3b68011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800faf0 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6800fc50 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b780c70b0 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68014440 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.983+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b780c7670 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.984+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b780a8e20 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.984+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3b680145a0 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.985+0000 7f3b5ffff640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b6406b2a0 0x7f3b6406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.985+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3b6808a100 con 0x7f3b780a8900 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.985+0000 7f3b777fe640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b6406b2a0 0x7f3b6406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.985+0000 7f3b777fe640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b6406b2a0 0x7f3b6406d760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c036000 tx=0x7f3b6c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:55.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:55.989+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b68055300 con 0x7f3b780a8900 2023-12-04T05:40:56.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.224+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) v1 -- 0x7f3b781303b0 con 0x7f3b780a8900 2023-12-04T05:40:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.265+0000 7f3b5ffff640 1 -- 172.21.15.111:0/1654995850 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v109) v1 ==== 144+0+0 (secure 0 0 0) 0x7f3b68058920 con 0x7f3b780a8900 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b6406b2a0 msgr2=0x7f3b6406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b6406b2a0 0x7f3b6406d760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c036000 tx=0x7f3b6c030000 comp rx=0 tx=0).stop 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 msgr2=0x7f3b781ee420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 0x7f3b781ee420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3b680119a0 tx=0x7f3b68011e60 comp rx=0 tx=0).stop 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 shutdown_connections 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f3b6406b2a0 0x7f3b6406d760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b781f2110 0x7f3b781f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b780a8900 0x7f3b781ee420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 --2- 172.21.15.111:0/1654995850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b60003970 0x7f3b780c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.267+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 >> 172.21.15.111:0/1654995850 conn(0x7f3b7806a4c0 msgr2=0x7f3b78143a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.268+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 shutdown_connections 2023-12-04T05:40:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.268+0000 7f3b7daee640 1 -- 172.21.15.111:0/1654995850 wait complete. 2023-12-04T05:40:56.280 INFO:tasks.cephfs_test_runner:That when a client unmounts, the thread count on the MDS goes back ... ok 2023-12-04T05:40:56.280 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts blocklisted client 2023-12-04T05:40:56.280 INFO:tasks.cephfs_test_runner:test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:40:56.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:56.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.378+0000 7f304ce5c640 1 Processor -- start 2023-12-04T05:40:56.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.378+0000 7f304ce5c640 1 -- start start 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f304ce5c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f304813cee0 0x7f304806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f304ce5c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f304815aba0 0x7f304806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f304ce5c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30480aea20 con 0x7f3048140200 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f304ce5c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304813d2c0 con 0x7f304815aba0 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f304ce5c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304815b3a0 con 0x7f304813cee0 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f3046d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f304815aba0 0x7f304806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f3045d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3048140200 0x7f304815a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37000/0 (socket says 172.21.15.111:37000) 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f3046d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f304815aba0 0x7f304806c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44122/0 (socket says 172.21.15.111:44122) 2023-12-04T05:40:56.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.379+0000 7f3045d9b640 1 -- 172.21.15.111:0/2323416091 learned_addr learned my addr 172.21.15.111:0/2323416091 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f3046d9d640 1 -- 172.21.15.111:0/2323416091 >> v2:172.21.15.192:3300/0 conn(0x7f304813cee0 msgr2=0x7f304806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f3046d9d640 1 --2- 172.21.15.111:0/2323416091 >> v2:172.21.15.192:3300/0 conn(0x7f304813cee0 0x7f304806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f3046d9d640 1 -- 172.21.15.111:0/2323416091 >> v1:172.21.15.111:6789/0 conn(0x7f3048140200 legacy=0x7f304815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f3046d9d640 1 -- 172.21.15.111:0/2323416091 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304815b0c0 con 0x7f304815aba0 2023-12-04T05:40:56.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f304559a640 1 -- 172.21.15.111:0/2323416091 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057059920 0 0) 0x7f30480aea20 con 0x7f3048140200 2023-12-04T05:40:56.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f3046d9d640 1 --2- 172.21.15.111:0/2323416091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304815aba0 0x7f304806c790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f30480b2280 tx=0x7f304002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f696917253fd994 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f304559a640 1 -- 172.21.15.111:0/2323416091 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304002e070 con 0x7f304815aba0 2023-12-04T05:40:56.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f304559a640 1 -- 172.21.15.111:0/2323416091 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30400315a0 con 0x7f304815aba0 2023-12-04T05:40:56.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.380+0000 7f304559a640 1 -- 172.21.15.111:0/2323416091 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040031930 con 0x7f304815aba0 2023-12-04T05:40:56.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 -- 172.21.15.111:0/2323416091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304815aba0 msgr2=0x7f304806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2323416091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304815aba0 0x7f304806c790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f30480b2280 tx=0x7f304002b840 comp rx=0 tx=0).stop 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 -- 172.21.15.111:0/2323416091 shutdown_connections 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2323416091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304815aba0 0x7f304806c790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2323416091 >> v2:172.21.15.192:3300/0 conn(0x7f304813cee0 0x7f304806c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 -- 172.21.15.111:0/2323416091 >> 172.21.15.111:0/2323416091 conn(0x7f304806a4c0 msgr2=0x7f304806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 -- 172.21.15.111:0/2323416091 shutdown_connections 2023-12-04T05:40:56.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.381+0000 7f304ce5c640 1 -- 172.21.15.111:0/2323416091 wait complete. 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.382+0000 7f304ce5c640 1 Processor -- start 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.382+0000 7f304ce5c640 1 -- start start 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304ce5c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 0x7f304812d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304ce5c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3048140200 0x7f3048153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304ce5c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304815aba0 0x7f30481515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304ce5c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048151af0 con 0x7f304813cee0 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304ce5c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30481f1810 con 0x7f304815aba0 2023-12-04T05:40:56.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304ce5c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30481f0ee0 con 0x7f3048140200 2023-12-04T05:40:56.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 0x7f304812d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 0x7f304812d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44124/0 (socket says 172.21.15.111:44124) 2023-12-04T05:40:56.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 -- 172.21.15.111:0/2352604186 learned_addr learned my addr 172.21.15.111:0/2352604186 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:56.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f3046d9d640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304815aba0 0x7f30481515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 -- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3048140200 msgr2=0x7f3048153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3048140200 0x7f3048153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 -- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304815aba0 msgr2=0x7f30481515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304815aba0 0x7f30481515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f304659c640 1 -- 172.21.15.111:0/2352604186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3040009400 con 0x7f304813cee0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.383+0000 7f3046d9d640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304815aba0 0x7f30481515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.384+0000 7f304659c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 0x7f304812d710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f303800ba50 tx=0x7f303800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.384+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800cbe0 con 0x7f304813cee0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.384+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303800cd60 con 0x7f304813cee0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.384+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038014680 con 0x7f304813cee0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.384+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30481f11c0 con 0x7f304813cee0 2023-12-04T05:40:56.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.384+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3048151d20 con 0x7f304813cee0 2023-12-04T05:40:56.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.385+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30480693d0 con 0x7f304813cee0 2023-12-04T05:40:56.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.385+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f303801d3b0 con 0x7f304813cee0 2023-12-04T05:40:56.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.386+0000 7f3036ffd640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f302806b2a0 0x7f302806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.386+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f30380898c0 con 0x7f304813cee0 2023-12-04T05:40:56.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.386+0000 7f3045d9b640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f302806b2a0 0x7f302806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.386+0000 7f3045d9b640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f302806b2a0 0x7f302806d760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f303c007fd0 tx=0x7f303c007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.390+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3038054a30 con 0x7f304813cee0 2023-12-04T05:40:56.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.559+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f304806d170 con 0x7f304813cee0 2023-12-04T05:40:56.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.560+0000 7f3036ffd640 1 -- 172.21.15.111:0/2352604186 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12962 (secure 0 0 0) 0x7f3038019020 con 0x7f304813cee0 2023-12-04T05:40:56.561 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:56.562 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":36,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:55.261200+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6837/975894294":"2023-12-05T05:40:52.139962+0000","172.21.15.111:6835/975894294":"2023-12-05T05:40:52.139962+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:56.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f302806b2a0 msgr2=0x7f302806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f302806b2a0 0x7f302806d760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f303c007fd0 tx=0x7f303c007920 comp rx=0 tx=0).stop 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 msgr2=0x7f304812d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 0x7f304812d710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f303800ba50 tx=0x7f303800bf10 comp rx=0 tx=0).stop 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 shutdown_connections 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f302806b2a0 0x7f302806d760 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304815aba0 0x7f30481515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3048140200 0x7f3048153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 --2- 172.21.15.111:0/2352604186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304813cee0 0x7f304812d710 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 >> 172.21.15.111:0/2352604186 conn(0x7f304806a4c0 msgr2=0x7f3048150870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 shutdown_connections 2023-12-04T05:40:56.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.562+0000 7f304ce5c640 1 -- 172.21.15.111:0/2352604186 wait complete. 2023-12-04T05:40:56.574 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:56.575 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:40:56.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.675+0000 7f62af656640 1 Processor -- start 2023-12-04T05:40:56.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.675+0000 7f62af656640 1 -- start start 2023-12-04T05:40:56.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62af656640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f62a813f640 0x7f62a813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62af656640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a8071d20 0x7f62a8072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62af656640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a813dd10 con 0x7f62a813cee0 2023-12-04T05:40:56.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62af656640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a8072670 con 0x7f62a813f640 2023-12-04T05:40:56.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62af656640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a813d2c0 con 0x7f62a8071d20 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62acbf1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f62a813f640 0x7f62a813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62ad3f2640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f62a813cee0 0x7f62a813d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:57132/0 (socket says 172.21.15.111:57132) 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62acbf1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f62a813f640 0x7f62a813fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:35520/0 (socket says 172.21.15.111:35520) 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.676+0000 7f62adbf3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a8071d20 0x7f62a8072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62ad3f2640 1 -- 172.21.15.111:0/757368948 learned_addr learned my addr 172.21.15.111:0/757368948 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62acbf1640 1 -- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a8071d20 msgr2=0x7f62a8072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62acbf1640 1 --2- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a8071d20 0x7f62a8072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62acbf1640 1 -- 172.21.15.111:0/757368948 >> v1:172.21.15.192:6789/0 conn(0x7f62a813cee0 legacy=0x7f62a813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62acbf1640 1 -- 172.21.15.111:0/757368948 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a81400e0 con 0x7f62a813f640 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.677+0000 7f62adbf3640 1 --2- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a8071d20 0x7f62a8072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:40:56.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62acbf1640 1 --2- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813f640 0x7f62a813fab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62980097a0 tx=0x7f629802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd9422819a8ebf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f629ffff640 1 -- 172.21.15.111:0/757368948 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629802f050 con 0x7f62a813f640 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f629ffff640 1 -- 172.21.15.111:0/757368948 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6298031700 con 0x7f62a813f640 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f629ffff640 1 -- 172.21.15.111:0/757368948 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298031a90 con 0x7f62a813f640 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62af656640 1 -- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813f640 msgr2=0x7f62a813fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62af656640 1 --2- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813f640 0x7f62a813fab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62980097a0 tx=0x7f629802b7a0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62af656640 1 -- 172.21.15.111:0/757368948 shutdown_connections 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62af656640 1 --2- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a8071d20 0x7f62a8072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62af656640 1 --2- 172.21.15.111:0/757368948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813f640 0x7f62a813fab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.678+0000 7f62af656640 1 -- 172.21.15.111:0/757368948 >> 172.21.15.111:0/757368948 conn(0x7f62a80b0cf0 msgr2=0x7f62a80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.679+0000 7f62af656640 1 -- 172.21.15.111:0/757368948 shutdown_connections 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.679+0000 7f62af656640 1 -- 172.21.15.111:0/757368948 wait complete. 2023-12-04T05:40:56.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 Processor -- start 2023-12-04T05:40:56.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 -- start start 2023-12-04T05:40:56.683 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 0x7f62a8134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813cee0 0x7f62a81529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a813f640 0x7f62a8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8154040 con 0x7f62a8071d20 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8153430 con 0x7f62a813cee0 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62af656640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8153710 con 0x7f62a813f640 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62ad3f2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 0x7f62a8134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62ad3f2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 0x7f62a8134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44134/0 (socket says 172.21.15.111:44134) 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.680+0000 7f62ad3f2640 1 -- 172.21.15.111:0/993947182 learned_addr learned my addr 172.21.15.111:0/993947182 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62acbf1640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813cee0 0x7f62a81529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62adbf3640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a813f640 0x7f62a8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62ad3f2640 1 -- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a813f640 msgr2=0x7f62a8158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62ad3f2640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a813f640 0x7f62a8158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62ad3f2640 1 -- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813cee0 msgr2=0x7f62a81529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62ad3f2640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813cee0 0x7f62a81529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62ad3f2640 1 -- 172.21.15.111:0/993947182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a0009400 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62ad3f2640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 0x7f62a8134180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f62a400cba0 tx=0x7f62a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a400ad40 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a4012400 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6298009400 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a400aea0 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.681+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a8153cd0 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.682+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a81391a0 con 0x7f62a8071d20 2023-12-04T05:40:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.683+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f62a4007070 con 0x7f62a8071d20 2023-12-04T05:40:56.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.683+0000 7f629dffb640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f628c06b2d0 0x7f628c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.683+0000 7f62acbf1640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f628c06b2d0 0x7f628c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.683+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f62a4089800 con 0x7f62a8071d20 2023-12-04T05:40:56.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.684+0000 7f62acbf1640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f628c06b2d0 0x7f628c06d790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6298035000 tx=0x7f6298008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.687+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62a40549a0 con 0x7f62a8071d20 2023-12-04T05:40:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.855+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62a8069360 con 0x7f62a8071d20 2023-12-04T05:40:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.855+0000 7f629dffb640 1 -- 172.21.15.111:0/993947182 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62a4057fc0 con 0x7f62a8071d20 2023-12-04T05:40:56.857 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f628c06b2d0 msgr2=0x7f628c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f628c06b2d0 0x7f628c06d790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6298035000 tx=0x7f6298008030 comp rx=0 tx=0).stop 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 msgr2=0x7f62a8134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 0x7f62a8134180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f62a400cba0 tx=0x7f62a400a430 comp rx=0 tx=0).stop 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 shutdown_connections 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f628c06b2d0 0x7f628c06d790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62a813f640 0x7f62a8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a813cee0 0x7f62a81529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 --2- 172.21.15.111:0/993947182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a8071d20 0x7f62a8134180 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 >> 172.21.15.111:0/993947182 conn(0x7f62a80b0cf0 msgr2=0x7f62a806cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 shutdown_connections 2023-12-04T05:40:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.857+0000 7f62af656640 1 -- 172.21.15.111:0/993947182 wait complete. 2023-12-04T05:40:56.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2023-12-04T05:40:56.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.967+0000 7f6dabbe3640 1 Processor -- start 2023-12-04T05:40:56.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.967+0000 7f6dabbe3640 1 -- start start 2023-12-04T05:40:56.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6dabbe3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da415a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6dabbe3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40aea20 con 0x7f6da413cee0 2023-12-04T05:40:56.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6dabbe3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da413d2c0 con 0x7f6da4140200 2023-12-04T05:40:56.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6dabbe3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da415b3a0 con 0x7f6da406c960 2023-12-04T05:40:56.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6da997f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6da413cee0 0x7f6da406c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37012/0 (socket says 172.21.15.111:37012) 2023-12-04T05:40:56.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6da997f640 1 -- 172.21.15.111:0/3862586427 learned_addr learned my addr 172.21.15.111:0/3862586427 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6da917e640 1 --2- 172.21.15.111:0/3862586427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614162268 0 0) 0x7f6da40aea20 con 0x7f6da413cee0 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.968+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d90003520 con 0x7f6da413cee0 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377858375 0 0) 0x7f6d9c001110 con 0x7f6da4140200 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da40aea20 con 0x7f6da4140200 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2428191536 0 0) 0x7f6d90003520 con 0x7f6da413cee0 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da917e640 1 -- 172.21.15.111:0/3862586427 >> v1:172.21.15.192:6789/0 conn(0x7f6da4140200 legacy=0x7f6da40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da917e640 1 -- 172.21.15.111:0/3862586427 >> v1:172.21.15.111:6789/0 conn(0x7f6da413cee0 legacy=0x7f6da406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da917e640 1 -- 172.21.15.111:0/3862586427 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da415b0c0 con 0x7f6da406c960 2023-12-04T05:40:56.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.969+0000 7f6da917e640 1 --2- 172.21.15.111:0/3862586427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da415a490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d940097d0 tx=0x7f6d9402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75863f3bab69062 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00019f0 con 0x7f6da406c960 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9402e050 con 0x7f6da406c960 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6da897d640 1 -- 172.21.15.111:0/3862586427 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94030730 con 0x7f6da406c960 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 -- 172.21.15.111:0/3862586427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 msgr2=0x7f6da415a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/3862586427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da415a490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d940097d0 tx=0x7f6d9402b770 comp rx=0 tx=0).stop 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 -- 172.21.15.111:0/3862586427 shutdown_connections 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/3862586427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da415a490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 -- 172.21.15.111:0/3862586427 >> 172.21.15.111:0/3862586427 conn(0x7f6da406a4c0 msgr2=0x7f6da406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 -- 172.21.15.111:0/3862586427 shutdown_connections 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.970+0000 7f6dabbe3640 1 -- 172.21.15.111:0/3862586427 wait complete. 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.971+0000 7f6dabbe3640 1 Processor -- start 2023-12-04T05:40:56.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.971+0000 7f6dabbe3640 1 -- start start 2023-12-04T05:40:56.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6dabbe3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da414a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6dabbe3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da413cee0 0x7f6da40b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6dabbe3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 0x7f6da4148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6dabbe3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4070a20 con 0x7f6da4140200 2023-12-04T05:40:56.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6dabbe3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da413f440 con 0x7f6da413cee0 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6dabbe3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40703f0 con 0x7f6da406c960 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 0x7f6da4148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 0x7f6da4148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44144/0 (socket says 172.21.15.111:44144) 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 -- 172.21.15.111:0/2889261780 learned_addr learned my addr 172.21.15.111:0/2889261780 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6da997f640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da414a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6da917e640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da413cee0 0x7f6da40b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 -- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 msgr2=0x7f6da414a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da414a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 -- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da413cee0 msgr2=0x7f6da40b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da413cee0 0x7f6da40b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6daa180640 1 -- 172.21.15.111:0/2889261780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d94009400 con 0x7f6da4140200 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6da997f640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.972+0000 7f6da917e640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da413cee0 0x7f6da40b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:56.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.973+0000 7f6daa180640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 0x7f6da4148060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00c3f0 tx=0x7f6d9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.973+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0052d0 con 0x7f6da4140200 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.973+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c005430 con 0x7f6da4140200 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.973+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c003830 con 0x7f6da4140200 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.973+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da4148ab0 con 0x7f6da4140200 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.973+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da4070640 con 0x7f6da4140200 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.974+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da4070b60 con 0x7f6da4140200 2023-12-04T05:40:56.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.974+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6d9c003990 con 0x7f6da4140200 2023-12-04T05:40:56.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.974+0000 7f6d9a7fc640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6d8806b2d0 0x7f6d8806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:56.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.975+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d9c088c20 con 0x7f6da4140200 2023-12-04T05:40:56.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.975+0000 7f6da997f640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6d8806b2d0 0x7f6d8806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:56.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.975+0000 7f6da997f640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6d8806b2d0 0x7f6d8806d790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6da00012e0 tx=0x7f6da0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:56.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:56.979+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6da4070b60 con 0x7f6da4140200 2023-12-04T05:40:57.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.210+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) v1 -- 0x7f6da413d2c0 con 0x7f6da4140200 2023-12-04T05:40:57.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.269+0000 7f6d9a7fc640 1 -- 172.21.15.111:0/2889261780 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v110) v1 ==== 156+0+0 (secure 0 0 0) 0x7f6da413d2c0 con 0x7f6da4140200 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6d8806b2d0 msgr2=0x7f6d8806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6d8806b2d0 0x7f6d8806d790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6da00012e0 tx=0x7f6da0009040 comp rx=0 tx=0).stop 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 msgr2=0x7f6da4148060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 0x7f6da4148060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00c3f0 tx=0x7f6d9c00c8b0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 shutdown_connections 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6d8806b2d0 0x7f6d8806d790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6da4140200 0x7f6da4148060 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da413cee0 0x7f6da40b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 --2- 172.21.15.111:0/2889261780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406c960 0x7f6da414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 >> 172.21.15.111:0/2889261780 conn(0x7f6da406a4c0 msgr2=0x7f6da415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:57.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 shutdown_connections 2023-12-04T05:40:57.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.271+0000 7f6dabbe3640 1 -- 172.21.15.111:0/2889261780 wait complete. 2023-12-04T05:40:57.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-04T05:40:57.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.385+0000 7fe5b3eec640 1 Processor -- start 2023-12-04T05:40:57.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.385+0000 7fe5b3eec640 1 -- start start 2023-12-04T05:40:57.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b3eec640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe5ac13cee0 0x7fe5ac13d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b3eec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13f640 0x7fe5ac13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b3eec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe5ac071d20 0x7fe5ac072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b3eec640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac0726b0 con 0x7fe5ac13cee0 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b3eec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac13d2c0 con 0x7fe5ac071d20 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b3eec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1400e0 con 0x7fe5ac13f640 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe5ac13cee0 0x7fe5ac13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe5ac13cee0 0x7fe5ac13d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44152/0 (socket says 172.21.15.111:44152) 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 -- 172.21.15.111:0/2660904484 learned_addr learned my addr 172.21.15.111:0/2660904484 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b2489640 1 --2- 172.21.15.111:0/2660904484 >> v2:172.21.15.192:3300/0 conn(0x7fe5ac071d20 0x7fe5ac072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1487640 1 --2- 172.21.15.111:0/2660904484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13f640 0x7fe5ac13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 -- 172.21.15.111:0/2660904484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13f640 msgr2=0x7fe5ac13fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 --2- 172.21.15.111:0/2660904484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13f640 0x7fe5ac13fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 -- 172.21.15.111:0/2660904484 >> v2:172.21.15.192:3300/0 conn(0x7fe5ac071d20 msgr2=0x7fe5ac072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 --2- 172.21.15.111:0/2660904484 >> v2:172.21.15.192:3300/0 conn(0x7fe5ac071d20 0x7fe5ac072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.386+0000 7fe5b1c88640 1 -- 172.21.15.111:0/2660904484 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac1403c0 con 0x7fe5ac13cee0 2023-12-04T05:40:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b1c88640 1 --2- 172.21.15.111:0/2660904484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13cee0 0x7fe5ac13d600 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0cc910 tx=0x7fe5a802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d1c06e1a1aa346 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b0c86640 1 -- 172.21.15.111:0/2660904484 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8031750 con 0x7fe5ac13cee0 2023-12-04T05:40:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b0c86640 1 -- 172.21.15.111:0/2660904484 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a80318b0 con 0x7fe5ac13cee0 2023-12-04T05:40:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b0c86640 1 -- 172.21.15.111:0/2660904484 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8031c40 con 0x7fe5ac13cee0 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b3eec640 1 -- 172.21.15.111:0/2660904484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13cee0 msgr2=0x7fe5ac13d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/2660904484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13cee0 0x7fe5ac13d600 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0cc910 tx=0x7fe5a802b7f0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b3eec640 1 -- 172.21.15.111:0/2660904484 shutdown_connections 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/2660904484 >> v2:172.21.15.192:3300/0 conn(0x7fe5ac071d20 0x7fe5ac072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/2660904484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13f640 0x7fe5ac13fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.387+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/2660904484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13cee0 0x7fe5ac13d600 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.388+0000 7fe5b3eec640 1 -- 172.21.15.111:0/2660904484 >> 172.21.15.111:0/2660904484 conn(0x7fe5ac0b0cf0 msgr2=0x7fe5ac0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.388+0000 7fe5b3eec640 1 -- 172.21.15.111:0/2660904484 shutdown_connections 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.388+0000 7fe5b3eec640 1 -- 172.21.15.111:0/2660904484 wait complete. 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 Processor -- start 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 -- start start 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ac071d20 0x7fe5ac142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13cee0 0x7fe5ac141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 0x7fe5ac150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac143010 con 0x7fe5ac13f640 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1655a0 con 0x7fe5ac071d20 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b3eec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac141650 con 0x7fe5ac13cee0 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.389+0000 7fe5b2489640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 0x7fe5ac150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 0x7fe5ac150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44168/0 (socket says 172.21.15.111:44168) 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 -- 172.21.15.111:0/1350657278 learned_addr learned my addr 172.21.15.111:0/1350657278 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b1c88640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ac071d20 0x7fe5ac142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b1487640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13cee0 0x7fe5ac141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 -- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13cee0 msgr2=0x7fe5ac141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13cee0 0x7fe5ac141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 -- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ac071d20 msgr2=0x7fe5ac142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ac071d20 0x7fe5ac142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 -- 172.21.15.111:0/1350657278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a80094d0 con 0x7fe5ac13f640 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b2489640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 0x7fe5ac150600 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4007e60 tx=0x7fe5a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4014420 con 0x7fe5ac13f640 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40145a0 con 0x7fe5ac13f640 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4014950 con 0x7fe5ac13f640 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac143220 con 0x7fe5ac13f640 2023-12-04T05:40:57.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.390+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac165230 con 0x7fe5ac13f640 2023-12-04T05:40:57.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.391+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac165710 con 0x7fe5ac13f640 2023-12-04T05:40:57.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.392+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe5a4014ab0 con 0x7fe5ac13f640 2023-12-04T05:40:57.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.392+0000 7fe5a27fc640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe59006b2a0 0x7fe59006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.392+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe5a408a220 con 0x7fe5ac13f640 2023-12-04T05:40:57.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.392+0000 7fe5b1c88640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe59006b2a0 0x7fe59006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.392+0000 7fe5b1c88640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe59006b2a0 0x7fe59006d760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8035000 tx=0x7fe5a8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:57.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.396+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5a4055390 con 0x7fe5ac13f640 2023-12-04T05:40:57.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.564+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe5ac069360 con 0x7fe5ac13f640 2023-12-04T05:40:57.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.567+0000 7fe5a27fc640 1 -- 172.21.15.111:0/1350657278 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5a40589b0 con 0x7fe5ac13f640 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe59006b2a0 msgr2=0x7fe59006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe59006b2a0 0x7fe59006d760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8035000 tx=0x7fe5a8008030 comp rx=0 tx=0).stop 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 msgr2=0x7fe5ac150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 0x7fe5ac150600 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4007e60 tx=0x7fe5a400e4b0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 shutdown_connections 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe59006b2a0 0x7fe59006d760 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5ac13f640 0x7fe5ac150600 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5ac13cee0 0x7fe5ac141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 --2- 172.21.15.111:0/1350657278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5ac071d20 0x7fe5ac142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 >> 172.21.15.111:0/1350657278 conn(0x7fe5ac0b0cf0 msgr2=0x7fe5ac133fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:57.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 shutdown_connections 2023-12-04T05:40:57.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.569+0000 7fe5b3eec640 1 -- 172.21.15.111:0/1350657278 wait complete. 2023-12-04T05:40:57.581 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-04T05:40:57.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:40:57.634 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:40:57.686 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-04T05:40:57.687 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:57.687 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:57.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:40:57.702 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:40:57.702 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:57.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:40:57.721 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:40:57.721 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:57.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:40:57.757 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:40:57.757 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:40:57.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:40:57.777 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-04T05:40:57.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:40:57.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.895+0000 7f208746b640 1 Processor -- start 2023-12-04T05:40:57.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.895+0000 7f208746b640 1 -- start start 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f208746b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f208746b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f208015dab0 0x7f20800703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f208746b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2080154220 con 0x7f208006cae0 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f208746b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208015d2f0 con 0x7f208015dab0 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f208746b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208015a5c0 con 0x7f20800b3310 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f2085a08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f208015dab0 0x7f20800703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f2085207640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f208006cae0 0x7f208006cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37024/0 (socket says 172.21.15.111:37024) 2023-12-04T05:40:57.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f2085207640 1 -- 172.21.15.111:0/2968039220 learned_addr learned my addr 172.21.15.111:0/2968039220 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:57.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f2085a08640 1 -- 172.21.15.111:0/2968039220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 msgr2=0x7f2080159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:57.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f2085a08640 1 --2- 172.21.15.111:0/2968039220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.896+0000 7f2085a08640 1 -- 172.21.15.111:0/2968039220 >> v1:172.21.15.111:6789/0 conn(0x7f208006cae0 legacy=0x7f208006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.897+0000 7f2077fff640 1 -- 172.21.15.111:0/2968039220 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994844067 0 0) 0x7f207c001490 con 0x7f208006cae0 2023-12-04T05:40:57.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.897+0000 7f2085a08640 1 -- 172.21.15.111:0/2968039220 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208015a8a0 con 0x7f208015dab0 2023-12-04T05:40:57.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.897+0000 7f2085a08640 1 --2- 172.21.15.111:0/2968039220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f20800703f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f20780097a0 tx=0x7f207802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfb12cebb357758 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:57.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.897+0000 7f2077fff640 1 -- 172.21.15.111:0/2968039220 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078031830 con 0x7f208015dab0 2023-12-04T05:40:57.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.897+0000 7f2077fff640 1 -- 172.21.15.111:0/2968039220 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2078031990 con 0x7f208015dab0 2023-12-04T05:40:57.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.897+0000 7f2077fff640 1 -- 172.21.15.111:0/2968039220 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078031d20 con 0x7f208015dab0 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 -- 172.21.15.111:0/2968039220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 msgr2=0x7f20800703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 --2- 172.21.15.111:0/2968039220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f20800703f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f20780097a0 tx=0x7f207802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 -- 172.21.15.111:0/2968039220 shutdown_connections 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 --2- 172.21.15.111:0/2968039220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f20800703f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 --2- 172.21.15.111:0/2968039220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 -- 172.21.15.111:0/2968039220 >> 172.21.15.111:0/2968039220 conn(0x7f208006a4c0 msgr2=0x7f208006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 -- 172.21.15.111:0/2968039220 shutdown_connections 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.898+0000 7f208746b640 1 -- 172.21.15.111:0/2968039220 wait complete. 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.899+0000 7f208746b640 1 Processor -- start 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.899+0000 7f208746b640 1 -- start start 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f208746b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f208006cae0 0x7f208014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f208746b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f208746b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f208014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f208746b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080167240 con 0x7f208015dab0 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f208746b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208014c3c0 con 0x7f208006cae0 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f208746b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208014c660 con 0x7f20800b3310 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f208014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f208014c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44180/0 (socket says 172.21.15.111:44180) 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 -- 172.21.15.111:0/1471306418 learned_addr learned my addr 172.21.15.111:0/1471306418 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085207640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f208006cae0 0x7f208014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2084a06640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 -- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 msgr2=0x7f2080148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 -- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f208006cae0 msgr2=0x7f208014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f208006cae0 0x7f208014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:57.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 -- 172.21.15.111:0/1471306418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009400 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085207640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f208006cae0 0x7f208014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2084a06640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.900+0000 7f2085a08640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f208014c890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2078034040 tx=0x7f20780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.901+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207803b9f0 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.901+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207803bb70 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.901+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080166910 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.901+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780453f0 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.901+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2080166ed0 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.902+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20800cc860 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.902+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f207804e450 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.903+0000 7f2075ffb640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f206406b2a0 0x7f206406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.903+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f20780aba50 con 0x7f208015dab0 2023-12-04T05:40:57.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.903+0000 7f2085207640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f206406b2a0 0x7f206406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:57.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.903+0000 7f2085207640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f206406b2a0 0x7f206406d760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f207c0016f0 tx=0x7f207c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:57.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:57.907+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2078076c50 con 0x7f208015dab0 2023-12-04T05:40:58.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.106+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20800693d0 con 0x7f208015dab0 2023-12-04T05:40:58.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.107+0000 7f2075ffb640 1 -- 172.21.15.111:0/1471306418 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f207807a270 con 0x7f208015dab0 2023-12-04T05:40:58.109 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:58.109 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/914975932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":914975932},{"type":"v1","addr":"172.21.15.192:6835","nonce":914975932}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/2722585661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2722585661},{"type":"v1","addr":"172.21.15.192:6834","nonce":2722585661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/4062340988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4062340988},{"type":"v1","addr":"172.21.15.111:6836","nonce":4062340988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2343648022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2343648022},{"type":"v1","addr":"172.21.15.111:6837","nonce":2343648022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2023-12-04T05:40:58.109 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 16 2023-12-04T05:40:58.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f206406b2a0 msgr2=0x7f206406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f206406b2a0 0x7f206406d760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f207c0016f0 tx=0x7f207c009040 comp rx=0 tx=0).stop 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 msgr2=0x7f208014c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f208014c890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2078034040 tx=0x7f20780046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 shutdown_connections 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f206406b2a0 0x7f206406d760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f208015dab0 0x7f208014c890 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20800b3310 0x7f2080148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 --2- 172.21.15.111:0/1471306418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f208006cae0 0x7f208014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 >> 172.21.15.111:0/1471306418 conn(0x7f208006a4c0 msgr2=0x7f208006cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 shutdown_connections 2023-12-04T05:40:58.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.109+0000 7f208746b640 1 -- 172.21.15.111:0/1471306418 wait complete. 2023-12-04T05:40:58.121 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-04T05:40:58.121 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-04T05:40:58.122 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-04T05:40:58.122 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:58.122 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-04T05:40:58.122 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-04T05:40:58.122 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-04T05:40:58.122 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:58.122 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-04T05:40:58.123 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-04T05:40:58.123 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-04T05:40:58.123 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:58.123 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-04T05:40:58.123 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-04T05:40:58.123 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-04T05:40:58.123 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:40:58.133 INFO:tasks.ceph.mds.a:Stopped 2023-12-04T05:40:58.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-04T05:40:58.134 INFO:tasks.ceph.mds.c:Stopped 2023-12-04T05:40:58.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-04T05:40:58.135 INFO:tasks.ceph.mds.b:Stopped 2023-12-04T05:40:58.135 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-04T05:40:58.136 INFO:tasks.ceph.mds.d:Stopped 2023-12-04T05:40:58.136 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-04T05:40:58.176 INFO:tasks.ceph.mds.a:Started 2023-12-04T05:40:58.176 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:58.177 INFO:tasks.ceph.mds.c:Started 2023-12-04T05:40:58.177 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:58.178 INFO:tasks.ceph.mds.b:Started 2023-12-04T05:40:58.178 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:58.179 INFO:tasks.ceph.mds.d:Started 2023-12-04T05:40:58.179 DEBUG:teuthology.parallel:result is None 2023-12-04T05:40:58.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-04T05:40:58.251 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2023-12-04T05:40:58.251 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2023-12-04T05:40:58.262 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-04T05:40:58.262 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-04T05:40:58.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.282+0000 7f5674701640 1 Processor -- start 2023-12-04T05:40:58.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.282+0000 7f5674701640 1 -- start start 2023-12-04T05:40:58.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.283+0000 7f5674701640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f566c0b20a0 0x7f566c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:58.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.283+0000 7f5674701640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566c0cdcb0 con 0x7f566c0b2ad0 2023-12-04T05:40:58.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.283+0000 7f5674701640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566c0d39e0 con 0x7f566c0d1780 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.283+0000 7f5674701640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c0d3cc0 con 0x7f566c0b20a0 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.283+0000 7f5672c9e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f566c0b2ad0 0x7f566c0ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37032/0 (socket says 172.21.15.111:37032) 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.283+0000 7f5672c9e640 1 -- 172.21.15.111:0/382821098 learned_addr learned my addr 172.21.15.111:0/382821098 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.284+0000 7f5671c9c640 1 --2- 172.21.15.111:0/382821098 >> v2:172.21.15.192:3300/0 conn(0x7f566c0b20a0 0x7f566c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.284+0000 7f567149b640 1 -- 172.21.15.111:0/382821098 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721424736 0 0) 0x7f566c0cdcb0 con 0x7f566c0b2ad0 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.284+0000 7f5671c9c640 1 -- 172.21.15.111:0/382821098 >> v1:172.21.15.192:6789/0 conn(0x7f566c0d1780 legacy=0x7f566c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.284+0000 7f5671c9c640 1 -- 172.21.15.111:0/382821098 >> v1:172.21.15.111:6789/0 conn(0x7f566c0b2ad0 legacy=0x7f566c0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.284+0000 7f5671c9c640 1 -- 172.21.15.111:0/382821098 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0cdf50 con 0x7f566c0b20a0 2023-12-04T05:40:58.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.285+0000 7f5671c9c640 1 --2- 172.21.15.111:0/382821098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b20a0 0x7f566c0d01e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f565c0097a0 tx=0x7f565c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b9b4220edb85e82 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.285+0000 7f567149b640 1 -- 172.21.15.111:0/382821098 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c031830 con 0x7f566c0b20a0 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.285+0000 7f567149b640 1 -- 172.21.15.111:0/382821098 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c031990 con 0x7f566c0b20a0 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.285+0000 7f567149b640 1 -- 172.21.15.111:0/382821098 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c031d20 con 0x7f566c0b20a0 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.285+0000 7f5674701640 1 -- 172.21.15.111:0/382821098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b20a0 msgr2=0x7f566c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.285+0000 7f5674701640 1 --2- 172.21.15.111:0/382821098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b20a0 0x7f566c0d01e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f565c0097a0 tx=0x7f565c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.286+0000 7f5674701640 1 -- 172.21.15.111:0/382821098 shutdown_connections 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.286+0000 7f5674701640 1 --2- 172.21.15.111:0/382821098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b20a0 0x7f566c0d01e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.286+0000 7f5674701640 1 -- 172.21.15.111:0/382821098 >> 172.21.15.111:0/382821098 conn(0x7f566c09c4e0 msgr2=0x7f566c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.286+0000 7f5674701640 1 -- 172.21.15.111:0/382821098 shutdown_connections 2023-12-04T05:40:58.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.286+0000 7f5674701640 1 -- 172.21.15.111:0/382821098 wait complete. 2023-12-04T05:40:58.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 Processor -- start 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 -- start start 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 0x7f566c0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b2ad0 0x7f566c14c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f566c0d1780 0x7f566c14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0cd1a0 con 0x7f566c0b20a0 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0ccbe0 con 0x7f566c0b2ad0 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.287+0000 7f5674701640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0ccec0 con 0x7f566c0d1780 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 0x7f566c0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 0x7f566c0c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44202/0 (socket says 172.21.15.111:44202) 2023-12-04T05:40:58.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 -- 172.21.15.111:0/685734005 learned_addr learned my addr 172.21.15.111:0/685734005 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5672c9e640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f566c0d1780 0x7f566c14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5671c9c640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b2ad0 0x7f566c14c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 -- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f566c0d1780 msgr2=0x7f566c14c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f566c0d1780 0x7f566c14c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 -- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b2ad0 msgr2=0x7f566c14c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b2ad0 0x7f566c14c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 -- 172.21.15.111:0/685734005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c009400 con 0x7f566c0b20a0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f567249d640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 0x7f566c0c9440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f566800c400 tx=0x7f566800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56680052d0 con 0x7f566c0b20a0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5668005430 con 0x7f566c0b20a0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c0b8590 con 0x7f566c0b20a0 2023-12-04T05:40:58.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668003830 con 0x7f566c0b20a0 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c14a080 con 0x7f566c0b20a0 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.289+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566c0b8ab0 con 0x7f566c0b20a0 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.290+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5668003990 con 0x7f566c0b20a0 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.290+0000 7f5662ffd640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f565006b2a0 0x7f565006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.290+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5668088e30 con 0x7f566c0b20a0 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.290+0000 7f5671c9c640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f565006b2a0 0x7f565006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:58.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.291+0000 7f5671c9c640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f565006b2a0 0x7f565006d760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f565c036000 tx=0x7f565c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:58.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.294+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5668053fc0 con 0x7f566c0b20a0 2023-12-04T05:40:58.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:58.474+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f566c0b83c0 con 0x7f566c0b20a0 2023-12-04T05:40:59.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.285+0000 7f5662ffd640 1 -- 172.21.15.111:0/685734005 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5668017020 con 0x7f566c0b20a0 2023-12-04T05:40:59.287 INFO:teuthology.orchestra.run.smithi111.stderr: removed all blocklist entries 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f565006b2a0 msgr2=0x7f565006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f565006b2a0 0x7f565006d760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f565c036000 tx=0x7f565c03c040 comp rx=0 tx=0).stop 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 msgr2=0x7f566c0c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 0x7f566c0c9440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f566800c400 tx=0x7f566800c8c0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 shutdown_connections 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f565006b2a0 0x7f565006d760 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f566c0d1780 0x7f566c14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f566c0b2ad0 0x7f566c14c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 --2- 172.21.15.111:0/685734005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566c0b20a0 0x7f566c0c9440 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 >> 172.21.15.111:0/685734005 conn(0x7f566c09c4e0 msgr2=0x7f566c0b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 shutdown_connections 2023-12-04T05:40:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.288+0000 7f5674701640 1 -- 172.21.15.111:0/685734005 wait complete. 2023-12-04T05:40:59.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-04T05:40:59.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.407+0000 7fd26849b640 1 Processor -- start 2023-12-04T05:40:59.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.407+0000 7fd26849b640 1 -- start start 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd26849b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd26015f4e0 0x7fd2600b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd26849b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2600cf9f0 0x7fd2600cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd26849b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2600cbca0 con 0x7fd26015b480 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd26849b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2600cbf40 con 0x7fd2600cf9f0 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd26849b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2600cc220 con 0x7fd26015f4e0 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2600cf9f0 0x7fd2600cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2600cf9f0 0x7fd2600cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44208/0 (socket says 172.21.15.111:44208) 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 -- 172.21.15.111:0/396298006 learned_addr learned my addr 172.21.15.111:0/396298006 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 -- 172.21.15.111:0/396298006 >> v2:172.21.15.192:3300/0 conn(0x7fd26015f4e0 msgr2=0x7fd2600b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 --2- 172.21.15.111:0/396298006 >> v2:172.21.15.192:3300/0 conn(0x7fd26015f4e0 0x7fd2600b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 -- 172.21.15.111:0/396298006 >> v1:172.21.15.192:6789/0 conn(0x7fd26015b480 legacy=0x7fd26015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:59.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.408+0000 7fd266a38640 1 -- 172.21.15.111:0/396298006 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600cc500 con 0x7fd2600cf9f0 2023-12-04T05:40:59.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.409+0000 7fd266a38640 1 --2- 172.21.15.111:0/396298006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2600cf9f0 0x7fd2600cca20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd260071b70 tx=0x7fd25802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e47635268b094b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:59.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.409+0000 7fd265235640 1 -- 172.21.15.111:0/396298006 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25802ca80 con 0x7fd2600cf9f0 2023-12-04T05:40:59.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.409+0000 7fd265235640 1 -- 172.21.15.111:0/396298006 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25802cbe0 con 0x7fd2600cf9f0 2023-12-04T05:40:59.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.409+0000 7fd265235640 1 -- 172.21.15.111:0/396298006 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580343f0 con 0x7fd2600cf9f0 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 -- 172.21.15.111:0/396298006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2600cf9f0 msgr2=0x7fd2600cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 --2- 172.21.15.111:0/396298006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2600cf9f0 0x7fd2600cca20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd260071b70 tx=0x7fd25802bee0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 -- 172.21.15.111:0/396298006 shutdown_connections 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 --2- 172.21.15.111:0/396298006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2600cf9f0 0x7fd2600cca20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 --2- 172.21.15.111:0/396298006 >> v2:172.21.15.192:3300/0 conn(0x7fd26015f4e0 0x7fd2600b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 -- 172.21.15.111:0/396298006 >> 172.21.15.111:0/396298006 conn(0x7fd2600b2a70 msgr2=0x7fd2600b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 -- 172.21.15.111:0/396298006 shutdown_connections 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.410+0000 7fd26849b640 1 -- 172.21.15.111:0/396298006 wait complete. 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.411+0000 7fd26849b640 1 Processor -- start 2023-12-04T05:40:59.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.411+0000 7fd26849b640 1 -- start start 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd26849b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2600cf9f0 0x7fd2601569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd26849b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 0x7fd2600b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd26849b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26015f4e0 0x7fd2600b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd26849b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600c9830 con 0x7fd26015b480 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd26849b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600b5890 con 0x7fd26015f4e0 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd26849b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600b5b70 con 0x7fd2600cf9f0 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 0x7fd2600b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 0x7fd2600b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44218/0 (socket says 172.21.15.111:44218) 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 -- 172.21.15.111:0/1521344667 learned_addr learned my addr 172.21.15.111:0/1521344667 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd266237640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2600cf9f0 0x7fd2601569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd266a38640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26015f4e0 0x7fd2600b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 -- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2600cf9f0 msgr2=0x7fd2601569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2600cf9f0 0x7fd2601569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 -- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26015f4e0 msgr2=0x7fd2600b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26015f4e0 0x7fd2600b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 -- 172.21.15.111:0/1521344667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258009b30 con 0x7fd26015b480 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd266a38640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26015f4e0 0x7fd2600b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd265a36640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 0x7fd2600b6930 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd25000b410 tx=0x7fd25000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.412+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250011830 con 0x7fd26015b480 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.413+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500119b0 con 0x7fd26015b480 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.413+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2600b5e50 con 0x7fd26015b480 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.413+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250011d60 con 0x7fd26015b480 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.413+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2600b6410 con 0x7fd26015b480 2023-12-04T05:40:59.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.414+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2600b6630 con 0x7fd26015b480 2023-12-04T05:40:59.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.414+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd25001a400 con 0x7fd26015b480 2023-12-04T05:40:59.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.414+0000 7fd256ffd640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd24406b2a0 0x7fd24406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.415+0000 7fd266237640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd24406b2a0 0x7fd24406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.415+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd250089930 con 0x7fd26015b480 2023-12-04T05:40:59.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.415+0000 7fd266237640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd24406b2a0 0x7fd24406d760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0017f0 tx=0x7fd25c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:59.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.419+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd250054af0 con 0x7fd26015b480 2023-12-04T05:40:59.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.648+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd260067460 con 0x7fd26015b480 2023-12-04T05:40:59.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.649+0000 7fd256ffd640 1 -- 172.21.15.111:0/1521344667 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd250015020 con 0x7fd26015b480 2023-12-04T05:40:59.650 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:59.650 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:40:59.650 INFO:teuthology.orchestra.run.smithi111.stdout: "auth_dump": [ 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.a", 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lpb6cJBAA7iF+mExHXiopk7JbrjCW1g==", 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.651 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.b", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1l+X3+MBAAQatwhQBZ5/VZ0xuNJpQSYA==", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.c", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lgo37KxAA/89DL0NuA+KyX+CUbzzU+A==", 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.652 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.d", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lPSdXOBAAGSdsougjKMP1afwkK/2gmw==", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.0", 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDOZW1lxDi6HRAA0pl+LbQDgGxT9fr9vGzWHA==", 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.1", 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDRZW1lu/JsHRAApgnvWoeQ9MAOPdBuFifdvA==", 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.654 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.2", 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDUZW1ldy+tGxAAhPf6cR9pyGpyD0cAy1nd5g==", 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.655 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.3", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDXZW1lDCs7GRAAADDCyZOB4a+fmmWEgezmxw==", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.4", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDdZW1lzq4XJRAAG90v2DSwzGMnGUEK14Z7ew==", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.656 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.5", 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDgZW1lvzQUJBAAYrLHQV/+N0irpVvSJh1LkQ==", 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.6", 2023-12-04T05:40:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDjZW1ll6sCIhAAlUiDCgD51TZDbjgznyHYSw==", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.7", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDmZW1ll2Z/HxAALDVVTrBx3aaiRVf/Is2p5Q==", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.658 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.0", 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1l+5T3ABAA7h+0Ksz9xnrfFZKgsBbhIA==", 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow r", 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.1", 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1lqipMChAAXB0Z40yv2jFsOlHuH8i4Pg==", 2023-12-04T05:40:59.659 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.2", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1lwejPBhAAVmTVqunSjtPrXqLWy4n1AA==", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:59.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.3", 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDLZW1lrQobEBAAAJ8wvchn2R7AvCfLR5Me0w==", 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.admin", 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1l+r3tBxAA2OQ02bhGgalFabd6uZY6Ww==", 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.x", 2023-12-04T05:40:59.662 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1l8LA1IRAAZmq/pSux9bgsmwY5LTOzIQ==", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.y", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQDKZW1lztwVHRAAHxYfbr/N8aCdSCIPzjqzXw==", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2023-12-04T05:40:59.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-12-04T05:40:59.664 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.664 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:40:59.664 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2023-12-04T05:40:59.664 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:40:59.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd24406b2a0 msgr2=0x7fd24406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd24406b2a0 0x7fd24406d760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0017f0 tx=0x7fd25c009040 comp rx=0 tx=0).stop 2023-12-04T05:40:59.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 msgr2=0x7fd2600b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 0x7fd2600b6930 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd25000b410 tx=0x7fd25000b8d0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 shutdown_connections 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd24406b2a0 0x7fd24406d760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26015f4e0 0x7fd2600b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd26015b480 0x7fd2600b6930 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 --2- 172.21.15.111:0/1521344667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2600cf9f0 0x7fd2601569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 >> 172.21.15.111:0/1521344667 conn(0x7fd2600b2a70 msgr2=0x7fd26006e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 shutdown_connections 2023-12-04T05:40:59.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.650+0000 7fd26849b640 1 -- 172.21.15.111:0/1521344667 wait complete. 2023-12-04T05:40:59.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:40:59.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.765+0000 7f14485fb640 1 Processor -- start 2023-12-04T05:40:59.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.765+0000 7f14485fb640 1 -- start start 2023-12-04T05:40:59.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f14485fb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f144006c960 0x7f144015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f14485fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1440140200 0x7f144015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f14485fb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400aea20 con 0x7f144013cee0 2023-12-04T05:40:59.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f14485fb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144013d2c0 con 0x7f144006c960 2023-12-04T05:40:59.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f14485fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144015b3a0 con 0x7f1440140200 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f144006c960 0x7f144015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f144006c960 0x7f144015a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44228/0 (socket says 172.21.15.111:44228) 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 -- 172.21.15.111:0/443147922 learned_addr learned my addr 172.21.15.111:0/443147922 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1446b98640 1 --2- 172.21.15.111:0/443147922 >> v2:172.21.15.192:3300/0 conn(0x7f1440140200 0x7f144015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 -- 172.21.15.111:0/443147922 >> v2:172.21.15.192:3300/0 conn(0x7f1440140200 msgr2=0x7f144015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 --2- 172.21.15.111:0/443147922 >> v2:172.21.15.192:3300/0 conn(0x7f1440140200 0x7f144015a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 -- 172.21.15.111:0/443147922 >> v1:172.21.15.192:6789/0 conn(0x7f144013cee0 legacy=0x7f144006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.766+0000 7f1445b96640 1 -- 172.21.15.111:0/443147922 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144015b0c0 con 0x7f144006c960 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.767+0000 7f1445b96640 1 --2- 172.21.15.111:0/443147922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144006c960 0x7f144015a490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f144006e8e0 tx=0x7f143002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f357d6a1ca3cbc93 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.767+0000 7f1445395640 1 -- 172.21.15.111:0/443147922 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143002f8e0 con 0x7f144006c960 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.767+0000 7f1445395640 1 -- 172.21.15.111:0/443147922 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143002fa40 con 0x7f144006c960 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.767+0000 7f1445395640 1 -- 172.21.15.111:0/443147922 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143002fdd0 con 0x7f144006c960 2023-12-04T05:40:59.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.767+0000 7f14485fb640 1 -- 172.21.15.111:0/443147922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144006c960 msgr2=0x7f144015a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.767+0000 7f14485fb640 1 --2- 172.21.15.111:0/443147922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144006c960 0x7f144015a490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f144006e8e0 tx=0x7f143002db20 comp rx=0 tx=0).stop 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.768+0000 7f14485fb640 1 -- 172.21.15.111:0/443147922 shutdown_connections 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.768+0000 7f14485fb640 1 --2- 172.21.15.111:0/443147922 >> v2:172.21.15.192:3300/0 conn(0x7f1440140200 0x7f144015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.768+0000 7f14485fb640 1 --2- 172.21.15.111:0/443147922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144006c960 0x7f144015a490 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.768+0000 7f14485fb640 1 -- 172.21.15.111:0/443147922 >> 172.21.15.111:0/443147922 conn(0x7f144006a4c0 msgr2=0x7f144006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.768+0000 7f14485fb640 1 -- 172.21.15.111:0/443147922 shutdown_connections 2023-12-04T05:40:59.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.768+0000 7f14485fb640 1 -- 172.21.15.111:0/443147922 wait complete. 2023-12-04T05:40:59.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.769+0000 7f14485fb640 1 Processor -- start 2023-12-04T05:40:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.769+0000 7f14485fb640 1 -- start start 2023-12-04T05:40:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.769+0000 7f14485fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f144006c960 0x7f144014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.769+0000 7f14485fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 0x7f144013d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.769+0000 7f14485fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440140200 0x7f14401363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f14485fb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144013e770 con 0x7f144013cee0 2023-12-04T05:40:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f14485fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144014c340 con 0x7f1440140200 2023-12-04T05:40:59.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f14485fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144013daf0 con 0x7f144006c960 2023-12-04T05:40:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 0x7f144013d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 0x7f144013d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44232/0 (socket says 172.21.15.111:44232) 2023-12-04T05:40:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 -- 172.21.15.111:0/3398224412 learned_addr learned my addr 172.21.15.111:0/3398224412 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:40:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 -- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f144006c960 msgr2=0x7f144014cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f144006c960 0x7f144014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 -- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440140200 msgr2=0x7f14401363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1446b98640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440140200 0x7f14401363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440140200 0x7f14401363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 -- 172.21.15.111:0/3398224412 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300094d0 con 0x7f144013cee0 2023-12-04T05:40:59.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1446b98640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440140200 0x7f14401363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1445b96640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 0x7f144013d460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f143002db60 tx=0x7f14300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430037d30 con 0x7f144013cee0 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430040420 con 0x7f144013cee0 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144013ddd0 con 0x7f144013cee0 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300407d0 con 0x7f144013cee0 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.770+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144013e390 con 0x7f144013cee0 2023-12-04T05:40:59.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.771+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400703d0 con 0x7f144013cee0 2023-12-04T05:40:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.772+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1430040930 con 0x7f144013cee0 2023-12-04T05:40:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.772+0000 7f1436ffd640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f142406b2a0 0x7f142406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:40:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.772+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1430033070 con 0x7f144013cee0 2023-12-04T05:40:59.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.772+0000 7f1446397640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f142406b2a0 0x7f142406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:40:59.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.772+0000 7f1446397640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f142406b2a0 0x7f142406d760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f143c000f60 tx=0x7f143c0079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:40:59.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.776+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14400703d0 con 0x7f144013cee0 2023-12-04T05:40:59.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.944+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f144013e8b0 con 0x7f144013cee0 2023-12-04T05:40:59.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.944+0000 7f1436ffd640 1 -- 172.21.15.111:0/3398224412 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f144013e8b0 con 0x7f144013cee0 2023-12-04T05:40:59.946 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:40:59.946 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":41,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:40:59.283182+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f142406b2a0 msgr2=0x7f142406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f142406b2a0 0x7f142406d760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f143c000f60 tx=0x7f143c0079d0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 msgr2=0x7f144013d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 0x7f144013d460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f143002db60 tx=0x7f14300046c0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 shutdown_connections 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f142406b2a0 0x7f142406d760 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440140200 0x7f14401363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f144013cee0 0x7f144013d460 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 --2- 172.21.15.111:0/3398224412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f144006c960 0x7f144014cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:40:59.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 >> 172.21.15.111:0/3398224412 conn(0x7f144006a4c0 msgr2=0x7f1440070840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:40:59.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 shutdown_connections 2023-12-04T05:40:59.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:40:59.946+0000 7f14485fb640 1 -- 172.21.15.111:0/3398224412 wait complete. 2023-12-04T05:40:59.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:40:59.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:41:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.060+0000 7f1ef4124640 1 Processor -- start 2023-12-04T05:41:00.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.060+0000 7f1ef4124640 1 -- start start 2023-12-04T05:41:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef4124640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1eec072430 0x7f1eec13fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef4124640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec13dd10 con 0x7f1eec13f640 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef4124640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec13d2c0 con 0x7f1eec13cee0 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef4124640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec1400e0 con 0x7f1eec072430 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef26c1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1eec072430 0x7f1eec13fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef16bf640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1eec13f640 0x7f1eec071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37036/0 (socket says 172.21.15.111:37036) 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef16bf640 1 -- 172.21.15.111:0/3990531004 learned_addr learned my addr 172.21.15.111:0/3990531004 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef26c1640 1 -- 172.21.15.111:0/3990531004 >> v1:172.21.15.192:6789/0 conn(0x7f1eec13cee0 legacy=0x7f1eec13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef26c1640 1 -- 172.21.15.111:0/3990531004 >> v1:172.21.15.111:6789/0 conn(0x7f1eec13f640 legacy=0x7f1eec071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef26c1640 1 -- 172.21.15.111:0/3990531004 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec1403c0 con 0x7f1eec072430 2023-12-04T05:41:00.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.061+0000 7f1ef0ebe640 1 -- 172.21.15.111:0/3990531004 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041554257 0 0) 0x7f1eec13dd10 con 0x7f1eec13f640 2023-12-04T05:41:00.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef26c1640 1 --2- 172.21.15.111:0/3990531004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec072430 0x7f1eec13fab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bfa0 tx=0x7f1ee402df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813033ec02831647 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef0ebe640 1 -- 172.21.15.111:0/3990531004 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee402e930 con 0x7f1eec072430 2023-12-04T05:41:00.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef0ebe640 1 -- 172.21.15.111:0/3990531004 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee402ea90 con 0x7f1eec072430 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef0ebe640 1 -- 172.21.15.111:0/3990531004 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee402ee20 con 0x7f1eec072430 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef4124640 1 -- 172.21.15.111:0/3990531004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec072430 msgr2=0x7f1eec13fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef4124640 1 --2- 172.21.15.111:0/3990531004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec072430 0x7f1eec13fab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bfa0 tx=0x7f1ee402df90 comp rx=0 tx=0).stop 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.062+0000 7f1ef4124640 1 -- 172.21.15.111:0/3990531004 shutdown_connections 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.063+0000 7f1ef4124640 1 --2- 172.21.15.111:0/3990531004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec072430 0x7f1eec13fab0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.063+0000 7f1ef4124640 1 -- 172.21.15.111:0/3990531004 >> 172.21.15.111:0/3990531004 conn(0x7f1eec0b0cf0 msgr2=0x7f1eec0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.063+0000 7f1ef4124640 1 -- 172.21.15.111:0/3990531004 shutdown_connections 2023-12-04T05:41:00.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.063+0000 7f1ef4124640 1 -- 172.21.15.111:0/3990531004 wait complete. 2023-12-04T05:41:00.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 Processor -- start 2023-12-04T05:41:00.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 -- start start 2023-12-04T05:41:00.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eec072430 0x7f1eec071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eec13cee0 0x7f1eec1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 0x7f1eec138670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec1599d0 con 0x7f1eec13f640 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec13b660 con 0x7f1eec13cee0 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.064+0000 7f1ef4124640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec140840 con 0x7f1eec072430 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 0x7f1eec138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 0x7f1eec138670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44260/0 (socket says 172.21.15.111:44260) 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 -- 172.21.15.111:0/2065550396 learned_addr learned my addr 172.21.15.111:0/2065550396 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef16bf640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eec13cee0 0x7f1eec1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef1ec0640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eec072430 0x7f1eec071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 -- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eec072430 msgr2=0x7f1eec071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eec072430 0x7f1eec071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 -- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eec13cee0 msgr2=0x7f1eec1f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eec13cee0 0x7f1eec1f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 -- 172.21.15.111:0/2065550396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee400bc80 con 0x7f1eec13f640 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef26c1640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 0x7f1eec138670 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bfa0 tx=0x7f1ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4036800 con 0x7f1eec13f640 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee4036980 con 0x7f1eec13f640 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec1f2fe0 con 0x7f1eec13f640 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4036d30 con 0x7f1eec13f640 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.065+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec1596f0 con 0x7f1eec13f640 2023-12-04T05:41:00.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.066+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec0ae850 con 0x7f1eec13f640 2023-12-04T05:41:00.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.067+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1ee404a460 con 0x7f1eec13f640 2023-12-04T05:41:00.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.067+0000 7f1ee27fc640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ed006b2a0 0x7f1ed006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.067+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1ee40aca10 con 0x7f1eec13f640 2023-12-04T05:41:00.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.067+0000 7f1ef1ec0640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ed006b2a0 0x7f1ed006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.068+0000 7f1ef1ec0640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ed006b2a0 0x7f1ed006d760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8006fd0 tx=0x7f1ee8009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.071+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ee4077da0 con 0x7f1eec13f640 2023-12-04T05:41:00.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.235+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1eec140a40 con 0x7f1eec13f640 2023-12-04T05:41:00.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.235+0000 7f1ee27fc640 1 -- 172.21.15.111:0/2065550396 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ee407b3c0 con 0x7f1eec13f640 2023-12-04T05:41:00.237 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ed006b2a0 msgr2=0x7f1ed006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ed006b2a0 0x7f1ed006d760 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8006fd0 tx=0x7f1ee8009060 comp rx=0 tx=0).stop 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 msgr2=0x7f1eec138670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 0x7f1eec138670 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bfa0 tx=0x7f1ee4004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 shutdown_connections 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1ed006b2a0 0x7f1ed006d760 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eec13f640 0x7f1eec138670 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1eec13cee0 0x7f1eec1f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 --2- 172.21.15.111:0/2065550396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1eec072430 0x7f1eec071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 >> 172.21.15.111:0/2065550396 conn(0x7f1eec0b0cf0 msgr2=0x7f1eec06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:00.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 shutdown_connections 2023-12-04T05:41:00.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.237+0000 7f1ef4124640 1 -- 172.21.15.111:0/2065550396 wait complete. 2023-12-04T05:41:00.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-04T05:41:00.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.347+0000 7ffb3f5c5640 1 Processor -- start 2023-12-04T05:41:00.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.347+0000 7ffb3f5c5640 1 -- start start 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3f5c5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffb3806cae0 0x7ffb3806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3f5c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb380b3310 0x7ffb38159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3f5c5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffb3806c250 0x7ffb3815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3f5c5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb38154220 con 0x7ffb3806cae0 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3f5c5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3815d2f0 con 0x7ffb3806c250 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3f5c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3806d400 con 0x7ffb380b3310 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3d361640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffb3806cae0 0x7ffb3806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3d361640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffb3806cae0 0x7ffb3806cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44266/0 (socket says 172.21.15.111:44266) 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.348+0000 7ffb3d361640 1 -- 172.21.15.111:0/473164846 learned_addr learned my addr 172.21.15.111:0/473164846 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3cb60640 1 --2- 172.21.15.111:0/473164846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb380b3310 0x7ffb38159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3db62640 1 --2- 172.21.15.111:0/473164846 >> v2:172.21.15.192:3300/0 conn(0x7ffb3806c250 0x7ffb3815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3d361640 1 -- 172.21.15.111:0/473164846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb380b3310 msgr2=0x7ffb38159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3d361640 1 --2- 172.21.15.111:0/473164846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb380b3310 0x7ffb38159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3d361640 1 -- 172.21.15.111:0/473164846 >> v2:172.21.15.192:3300/0 conn(0x7ffb3806c250 msgr2=0x7ffb3815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3d361640 1 --2- 172.21.15.111:0/473164846 >> v2:172.21.15.192:3300/0 conn(0x7ffb3806c250 0x7ffb3815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3d361640 1 -- 172.21.15.111:0/473164846 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3815a5c0 con 0x7ffb3806cae0 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3db62640 1 --2- 172.21.15.111:0/473164846 >> v2:172.21.15.192:3300/0 conn(0x7ffb3806c250 0x7ffb3815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb3d361640 1 --2- 172.21.15.111:0/473164846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3806cae0 0x7ffb3806cec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffb380b3090 tx=0x7ffb2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764135298d0f5c0a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb37fff640 1 -- 172.21.15.111:0/473164846 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2802ca70 con 0x7ffb3806cae0 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.349+0000 7ffb37fff640 1 -- 172.21.15.111:0/473164846 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2802cbd0 con 0x7ffb3806cae0 2023-12-04T05:41:00.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb37fff640 1 -- 172.21.15.111:0/473164846 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280343f0 con 0x7ffb3806cae0 2023-12-04T05:41:00.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/473164846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3806cae0 msgr2=0x7ffb3806cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/473164846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3806cae0 0x7ffb3806cec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffb380b3090 tx=0x7ffb2802bed0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/473164846 shutdown_connections 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/473164846 >> v2:172.21.15.192:3300/0 conn(0x7ffb3806c250 0x7ffb3815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/473164846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb380b3310 0x7ffb38159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/473164846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3806cae0 0x7ffb3806cec0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/473164846 >> 172.21.15.111:0/473164846 conn(0x7ffb3806a4c0 msgr2=0x7ffb3806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/473164846 shutdown_connections 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.350+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/473164846 wait complete. 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.351+0000 7ffb3f5c5640 1 Processor -- start 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.351+0000 7ffb3f5c5640 1 -- start start 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3f5c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb3806c250 0x7ffb3814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3f5c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb3806cae0 0x7ffb38158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3f5c5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 0x7ffb38162670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3f5c5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38150200 con 0x7ffb380b3310 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3f5c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380b0ad0 con 0x7ffb3806cae0 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3f5c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380af7e0 con 0x7ffb3806c250 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 0x7ffb38162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 0x7ffb38162670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44282/0 (socket says 172.21.15.111:44282) 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 -- 172.21.15.111:0/3439269660 learned_addr learned my addr 172.21.15.111:0/3439269660 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3cb60640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb3806cae0 0x7ffb38158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3d361640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb3806c250 0x7ffb3814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 -- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb3806c250 msgr2=0x7ffb3814fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb3806c250 0x7ffb3814fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 -- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb3806cae0 msgr2=0x7ffb38158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb3806cae0 0x7ffb38158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3db62640 1 -- 172.21.15.111:0/3439269660 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28009b30 con 0x7ffb380b3310 2023-12-04T05:41:00.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3d361640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb3806c250 0x7ffb3814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.352+0000 7ffb3cb60640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb3806cae0 0x7ffb38158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.353+0000 7ffb3db62640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 0x7ffb38162670 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c7c0 tx=0x7ffb3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.353+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30004230 con 0x7ffb380b3310 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.353+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30004390 con 0x7ffb380b3310 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.353+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30007860 con 0x7ffb380b3310 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.353+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb380afa30 con 0x7ffb380b3310 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.353+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb380afff0 con 0x7ffb380b3310 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.354+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb380693d0 con 0x7ffb380b3310 2023-12-04T05:41:00.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.354+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ffb300079c0 con 0x7ffb380b3310 2023-12-04T05:41:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.355+0000 7ffb35ffb640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb2406b2d0 0x7ffb2406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.355+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffb30089a90 con 0x7ffb380b3310 2023-12-04T05:41:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.355+0000 7ffb3d361640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb2406b2d0 0x7ffb2406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.355+0000 7ffb3d361640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb2406b2d0 0x7ffb2406d790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb28035000 tx=0x7ffb28030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.359+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb30054d10 con 0x7ffb380b3310 2023-12-04T05:41:00.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.526+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffb38071900 con 0x7ffb380b3310 2023-12-04T05:41:00.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.526+0000 7ffb35ffb640 1 -- 172.21.15.111:0/3439269660 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffb30058330 con 0x7ffb380b3310 2023-12-04T05:41:00.528 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:00.528 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2023-12-04T05:41:00.528 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-12-04T05:41:00.529 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2023-12-04T05:41:00.529 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2023-12-04T05:41:00.529 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:41:00.529 INFO:teuthology.orchestra.run.smithi111.stdout:] 2023-12-04T05:41:00.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb2406b2d0 msgr2=0x7ffb2406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb2406b2d0 0x7ffb2406d790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb28035000 tx=0x7ffb28030000 comp rx=0 tx=0).stop 2023-12-04T05:41:00.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 msgr2=0x7ffb38162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 0x7ffb38162670 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c7c0 tx=0x7ffb3000cc80 comp rx=0 tx=0).stop 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 shutdown_connections 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb2406b2d0 0x7ffb2406d790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb380b3310 0x7ffb38162670 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb3806cae0 0x7ffb38158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 --2- 172.21.15.111:0/3439269660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb3806c250 0x7ffb3814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 >> 172.21.15.111:0/3439269660 conn(0x7ffb3806a4c0 msgr2=0x7ffb380b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 shutdown_connections 2023-12-04T05:41:00.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.528+0000 7ffb3f5c5640 1 -- 172.21.15.111:0/3439269660 wait complete. 2023-12-04T05:41:00.540 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-04T05:41:00.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-04T05:41:00.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.642+0000 7f6bc6190640 1 Processor -- start 2023-12-04T05:41:00.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.642+0000 7f6bc6190640 1 -- start start 2023-12-04T05:41:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.643+0000 7f6bc6190640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 0x7f6bc00d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.643+0000 7f6bc6190640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6bc00d10f0 0x7f6bc00d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.643+0000 7f6bc6190640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc00cfaf0 con 0x7f6bc00d1cb0 2023-12-04T05:41:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.643+0000 7f6bc6190640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00d2140 con 0x7f6bc00d10f0 2023-12-04T05:41:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.643+0000 7f6bc6190640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00d1530 con 0x7f6bc015d480 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbeffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6bc00d10f0 0x7f6bc00d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 0x7f6bc00d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 0x7f6bc00d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:36042/0 (socket says 172.21.15.111:36042) 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 -- 172.21.15.111:0/3923884641 learned_addr learned my addr 172.21.15.111:0/3923884641 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbeffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6bc00d10f0 0x7f6bc00d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44120/0 (socket says 172.21.15.111:44120) 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 -- 172.21.15.111:0/3923884641 >> v2:172.21.15.192:3300/0 conn(0x7f6bc00d10f0 msgr2=0x7f6bc00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 --2- 172.21.15.111:0/3923884641 >> v2:172.21.15.192:3300/0 conn(0x7f6bc00d10f0 0x7f6bc00d1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 -- 172.21.15.111:0/3923884641 >> v1:172.21.15.192:6789/0 conn(0x7f6bc00d1cb0 legacy=0x7f6bc00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.644+0000 7f6bbf7fe640 1 -- 172.21.15.111:0/3923884641 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00d3fa0 con 0x7f6bc015d480 2023-12-04T05:41:00.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bbf7fe640 1 --2- 172.21.15.111:0/3923884641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 0x7f6bc00d38a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b639cc70420c37 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bbe7fc640 1 -- 172.21.15.111:0/3923884641 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0030840 con 0x7f6bc015d480 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bbe7fc640 1 -- 172.21.15.111:0/3923884641 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb00309a0 con 0x7f6bc015d480 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bbe7fc640 1 -- 172.21.15.111:0/3923884641 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0030d30 con 0x7f6bc015d480 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bc6190640 1 -- 172.21.15.111:0/3923884641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 msgr2=0x7f6bc00d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bc6190640 1 --2- 172.21.15.111:0/3923884641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 0x7f6bc00d38a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb002b980 comp rx=0 tx=0).stop 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bc6190640 1 -- 172.21.15.111:0/3923884641 shutdown_connections 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bc6190640 1 --2- 172.21.15.111:0/3923884641 >> v2:172.21.15.192:3300/0 conn(0x7f6bc00d10f0 0x7f6bc00d1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bc6190640 1 --2- 172.21.15.111:0/3923884641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc015d480 0x7f6bc00d38a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.645+0000 7f6bc6190640 1 -- 172.21.15.111:0/3923884641 >> 172.21.15.111:0/3923884641 conn(0x7f6bc009c4d0 msgr2=0x7f6bc00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.646+0000 7f6bc6190640 1 -- 172.21.15.111:0/3923884641 shutdown_connections 2023-12-04T05:41:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.646+0000 7f6bc6190640 1 -- 172.21.15.111:0/3923884641 wait complete. 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.647+0000 7f6bc6190640 1 Processor -- start 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.647+0000 7f6bc6190640 1 -- start start 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc00d10f0 0x7f6bc015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc00d1cb0 0x7f6bc00ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 0x7f6bc0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cc730 con 0x7f6bc015d480 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cc170 con 0x7f6bc00d10f0 2023-12-04T05:41:00.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cc450 con 0x7f6bc00d1cb0 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 0x7f6bc0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 0x7f6bc0152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57756/0 (socket says 172.21.15.111:57756) 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 -- 172.21.15.111:0/2474183171 learned_addr learned my addr 172.21.15.111:0/2474183171 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbeffd640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc00d1cb0 0x7f6bc00ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbf7fe640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc00d10f0 0x7f6bc015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 -- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc00d1cb0 msgr2=0x7f6bc00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc00d1cb0 0x7f6bc00ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 -- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc00d10f0 msgr2=0x7f6bc015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc00d10f0 0x7f6bc015dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 -- 172.21.15.111:0/2474183171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00094d0 con 0x7f6bc015d480 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bbffff640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 0x7f6bc0152440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c7a0 tx=0x7f6bb800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8003b70 con 0x7f6bc015d480 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb8003cf0 con 0x7f6bc015d480 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc00b5e50 con 0x7f6bc015d480 2023-12-04T05:41:00.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8003ea0 con 0x7f6bc015d480 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.648+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc00cd310 con 0x7f6bc015d480 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.649+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00b4af0 con 0x7f6bc015d480 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.650+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6bb800aa90 con 0x7f6bc015d480 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.650+0000 7f6ba7fff640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6bac06b2a0 0x7f6bac06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.650+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bb8089790 con 0x7f6bc015d480 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.650+0000 7f6bbf7fe640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6bac06b2a0 0x7f6bac06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.651+0000 7f6bbf7fe640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6bac06b2a0 0x7f6bac06d760 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:00.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.655+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bb80548d0 con 0x7f6bc015d480 2023-12-04T05:41:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:00.824+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6bc00b6370 con 0x7f6bc015d480 2023-12-04T05:41:01.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.290+0000 7f6ba7fff640 1 -- 172.21.15.111:0/2474183171 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6bc00b6370 con 0x7f6bc015d480 2023-12-04T05:41:01.292 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2023-12-04T05:41:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6bac06b2a0 msgr2=0x7f6bac06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6bac06b2a0 0x7f6bac06d760 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0036000 tx=0x7f6bb0008030 comp rx=0 tx=0).stop 2023-12-04T05:41:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 msgr2=0x7f6bc0152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 0x7f6bc0152440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c7a0 tx=0x7f6bb800cc60 comp rx=0 tx=0).stop 2023-12-04T05:41:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 shutdown_connections 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6bac06b2a0 0x7f6bac06d760 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6bc015d480 0x7f6bc0152440 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc00d1cb0 0x7f6bc00ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 --2- 172.21.15.111:0/2474183171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc00d10f0 0x7f6bc015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 >> 172.21.15.111:0/2474183171 conn(0x7f6bc009c4d0 msgr2=0x7f6bc006bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 shutdown_connections 2023-12-04T05:41:01.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.293+0000 7f6bc6190640 1 -- 172.21.15.111:0/2474183171 wait complete. 2023-12-04T05:41:01.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-04T05:41:01.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.413+0000 7fdedcda9640 1 Processor -- start 2023-12-04T05:41:01.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.413+0000 7fdedcda9640 1 -- start start 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fdedcda9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80b4c50 0x7fded80b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fdedcda9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fded80ce370 0x7fded80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fdedcda9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded80b5570 con 0x7fded815e9a0 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fdedcda9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded80d1530 con 0x7fded80ce370 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fdedcda9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded80d1810 con 0x7fded80b4c50 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fded6d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fded80ce370 0x7fded80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fded6d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fded80ce370 0x7fded80b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57770/0 (socket says 172.21.15.111:57770) 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fded6d9d640 1 -- 172.21.15.111:0/2130628516 learned_addr learned my addr 172.21.15.111:0/2130628516 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.414+0000 7fdecdd9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fded815e9a0 0x7fded80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39566/0 (socket says 172.21.15.111:39566) 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded659c640 1 --2- 172.21.15.111:0/2130628516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80b4c50 0x7fded80b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded6d9d640 1 -- 172.21.15.111:0/2130628516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80b4c50 msgr2=0x7fded80b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:01.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded6d9d640 1 --2- 172.21.15.111:0/2130628516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80b4c50 0x7fded80b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded6d9d640 1 -- 172.21.15.111:0/2130628516 >> v1:172.21.15.111:6789/0 conn(0x7fded815e9a0 legacy=0x7fded80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:01.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded6d9d640 1 -- 172.21.15.111:0/2130628516 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80d1af0 con 0x7fded80ce370 2023-12-04T05:41:01.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded6d9d640 1 --2- 172.21.15.111:0/2130628516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded80ce370 0x7fded80b1c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fded0009e50 tx=0x7fded002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63c77837ea58e38 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:01.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.415+0000 7fded5d9b640 1 -- 172.21.15.111:0/2130628516 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded002c9e0 con 0x7fded80ce370 2023-12-04T05:41:01.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fded5d9b640 1 -- 172.21.15.111:0/2130628516 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded002cb40 con 0x7fded80ce370 2023-12-04T05:41:01.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fded5d9b640 1 -- 172.21.15.111:0/2130628516 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00343f0 con 0x7fded80ce370 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 -- 172.21.15.111:0/2130628516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded80ce370 msgr2=0x7fded80b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2130628516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded80ce370 0x7fded80b1c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fded0009e50 tx=0x7fded002be40 comp rx=0 tx=0).stop 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 -- 172.21.15.111:0/2130628516 shutdown_connections 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2130628516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded80ce370 0x7fded80b1c40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2130628516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80b4c50 0x7fded80b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 -- 172.21.15.111:0/2130628516 >> 172.21.15.111:0/2130628516 conn(0x7fded8072220 msgr2=0x7fded80cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 -- 172.21.15.111:0/2130628516 shutdown_connections 2023-12-04T05:41:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.416+0000 7fdedcda9640 1 -- 172.21.15.111:0/2130628516 wait complete. 2023-12-04T05:41:01.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 Processor -- start 2023-12-04T05:41:01.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 -- start start 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded80b4c50 0x7fded80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80ce370 0x7fded80cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 0x7fded80b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded80c89b0 con 0x7fded815e9a0 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded80b6390 con 0x7fded80b4c50 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fdedcda9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded80c86d0 con 0x7fded80ce370 2023-12-04T05:41:01.422 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fded6d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 0x7fded80b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fded6d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 0x7fded80b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57774/0 (socket says 172.21.15.111:57774) 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.418+0000 7fded6d9d640 1 -- 172.21.15.111:0/2066241046 learned_addr learned my addr 172.21.15.111:0/2066241046 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded659c640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded80b4c50 0x7fded80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdecdd9b640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80ce370 0x7fded80cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded6d9d640 1 -- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80ce370 msgr2=0x7fded80cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded6d9d640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80ce370 0x7fded80cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded6d9d640 1 -- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded80b4c50 msgr2=0x7fded80cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded6d9d640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded80b4c50 0x7fded80cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded6d9d640 1 -- 172.21.15.111:0/2066241046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0009b30 con 0x7fded815e9a0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded659c640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded80b4c50 0x7fded80cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdecdd9b640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80ce370 0x7fded80cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fded6d9d640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 0x7fded80b6e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fded0036040 tx=0x7fded0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded003e450 con 0x7fded815e9a0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded003e5d0 con 0x7fded815e9a0 2023-12-04T05:41:01.423 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded80b6630 con 0x7fded815e9a0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded003e980 con 0x7fded815e9a0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.419+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded8150750 con 0x7fded815e9a0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.420+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded80b6b50 con 0x7fded815e9a0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.420+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fded003eae0 con 0x7fded815e9a0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.421+0000 7fdecf7fe640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdeb806b2a0 0x7fdeb806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.421+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fded00ac810 con 0x7fded815e9a0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.421+0000 7fded659c640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdeb806b2a0 0x7fdeb806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:01.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.421+0000 7fded659c640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdeb806b2a0 0x7fdeb806d760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006fd0 tx=0x7fdec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:01.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.425+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fded80b6b50 con 0x7fded815e9a0 2023-12-04T05:41:01.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:01.593+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fded80b6b50 con 0x7fded815e9a0 2023-12-04T05:41:02.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.297+0000 7fdecf7fe640 1 -- 172.21.15.111:0/2066241046 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7fded80b6b50 con 0x7fded815e9a0 2023-12-04T05:41:02.299 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdeb806b2a0 msgr2=0x7fdeb806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdeb806b2a0 0x7fdeb806d760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006fd0 tx=0x7fdec8008040 comp rx=0 tx=0).stop 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 msgr2=0x7fded80b6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 0x7fded80b6e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fded0036040 tx=0x7fded0004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 shutdown_connections 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdeb806b2a0 0x7fdeb806d760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fded815e9a0 0x7fded80b6e30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fded80ce370 0x7fded80cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.299+0000 7fdedcda9640 1 --2- 172.21.15.111:0/2066241046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fded80b4c50 0x7fded80cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.300+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 >> 172.21.15.111:0/2066241046 conn(0x7fded8072220 msgr2=0x7fded815ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.300+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 shutdown_connections 2023-12-04T05:41:02.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.300+0000 7fdedcda9640 1 -- 172.21.15.111:0/2066241046 wait complete. 2023-12-04T05:41:02.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-04T05:41:02.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.428+0000 7f22eac64640 1 Processor -- start 2023-12-04T05:41:02.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.428+0000 7f22eac64640 1 -- start start 2023-12-04T05:41:02.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22eac64640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e413d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:02.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22eac64640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22e413f640 0x7f22e413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:02.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22eac64640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e413d2c0 con 0x7f22e4071d20 2023-12-04T05:41:02.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22eac64640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e41400e0 con 0x7f22e413f640 2023-12-04T05:41:02.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22eac64640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41403c0 con 0x7f22e413cee0 2023-12-04T05:41:02.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22e413f640 0x7f22e413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:02.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22e413f640 0x7f22e413fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57776/0 (socket says 172.21.15.111:57776) 2023-12-04T05:41:02.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 -- 172.21.15.111:0/4164338194 learned_addr learned my addr 172.21.15.111:0/4164338194 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:02.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e8a00640 1 --2- 172.21.15.111:0/4164338194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:02.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 -- 172.21.15.111:0/4164338194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 msgr2=0x7f22e413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:02.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 --2- 172.21.15.111:0/4164338194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e413d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 -- 172.21.15.111:0/4164338194 >> v1:172.21.15.111:6789/0 conn(0x7f22e4071d20 legacy=0x7f22e4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:02.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.429+0000 7f22e3fff640 1 -- 172.21.15.111:0/4164338194 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40cdb50 con 0x7f22e413f640 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.430+0000 7f22e3fff640 1 --2- 172.21.15.111:0/4164338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e413f640 0x7f22e413fa80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f22d000bb10 tx=0x7f22d002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5974792c774f69 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.430+0000 7f22e37fe640 1 -- 172.21.15.111:0/4164338194 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d002ede0 con 0x7f22e413f640 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.430+0000 7f22e37fe640 1 -- 172.21.15.111:0/4164338194 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0036440 con 0x7f22e413f640 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.430+0000 7f22e37fe640 1 -- 172.21.15.111:0/4164338194 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00367d0 con 0x7f22e413f640 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 -- 172.21.15.111:0/4164338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e413f640 msgr2=0x7f22e413fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 --2- 172.21.15.111:0/4164338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e413f640 0x7f22e413fa80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f22d000bb10 tx=0x7f22d002e030 comp rx=0 tx=0).stop 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 -- 172.21.15.111:0/4164338194 shutdown_connections 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 --2- 172.21.15.111:0/4164338194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e413f640 0x7f22e413fa80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 --2- 172.21.15.111:0/4164338194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 -- 172.21.15.111:0/4164338194 >> 172.21.15.111:0/4164338194 conn(0x7f22e40b0cf0 msgr2=0x7f22e40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:02.436 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 -- 172.21.15.111:0/4164338194 shutdown_connections 2023-12-04T05:41:02.436 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.431+0000 7f22eac64640 1 -- 172.21.15.111:0/4164338194 wait complete. 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.432+0000 7f22eac64640 1 Processor -- start 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.432+0000 7f22eac64640 1 -- start start 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22eac64640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 0x7f22e41f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22eac64640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22eac64640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e413f640 0x7f22e414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22eac64640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413ba60 con 0x7f22e4071d20 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22eac64640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413ae90 con 0x7f22e413f640 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22eac64640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413b130 con 0x7f22e413cee0 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 0x7f22e41f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 0x7f22e41f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57778/0 (socket says 172.21.15.111:57778) 2023-12-04T05:41:02.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 -- 172.21.15.111:0/3161896852 learned_addr learned my addr 172.21.15.111:0/3161896852 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e3fff640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 -- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 msgr2=0x7f22e414e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e414e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e9201640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e413f640 0x7f22e414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 -- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e413f640 msgr2=0x7f22e414e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e413f640 0x7f22e414e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 -- 172.21.15.111:0/3161896852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d000b770 con 0x7f22e4071d20 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e9201640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e413f640 0x7f22e414e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e8a00640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 0x7f22e41f5e20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f22d800d800 tx=0x7f22d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:02.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.433+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800b6c0 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.434+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d800b840 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.434+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e413b410 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.434+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800bbf0 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.434+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e41f6360 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.434+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e413bed0 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.435+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f22d800bd50 con 0x7f22e4071d20 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.435+0000 7f22e17fa640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f22c806b2d0 0x7f22c806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.436+0000 7f22e3fff640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f22c806b2d0 0x7f22c806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:02.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.436+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f22d808a120 con 0x7f22e4071d20 2023-12-04T05:41:02.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.436+0000 7f22e3fff640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f22c806b2d0 0x7f22c806d790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f22d0038000 tx=0x7f22d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:02.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.440+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22e413bed0 con 0x7f22e4071d20 2023-12-04T05:41:02.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:02.642+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f22e413bed0 con 0x7f22e4071d20 2023-12-04T05:41:03.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.311+0000 7f22e17fa640 1 -- 172.21.15.111:0/3161896852 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v18) v1 ==== 172+0+0 (secure 0 0 0) 0x7f22e413bed0 con 0x7f22e4071d20 2023-12-04T05:41:03.313 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 6 and data pool 7 2023-12-04T05:41:03.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.319+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f22c806b2d0 msgr2=0x7f22c806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.319+0000 7f22eac64640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f22c806b2d0 0x7f22c806d790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f22d0038000 tx=0x7f22d0008040 comp rx=0 tx=0).stop 2023-12-04T05:41:03.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.319+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 msgr2=0x7f22e41f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.319+0000 7f22eac64640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 0x7f22e41f5e20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f22d800d800 tx=0x7f22d800dcc0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 shutdown_connections 2023-12-04T05:41:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f22c806b2d0 0x7f22c806d790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e413f640 0x7f22e414e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e413cee0 0x7f22e414e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 --2- 172.21.15.111:0/3161896852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e4071d20 0x7f22e41f5e20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 >> 172.21.15.111:0/3161896852 conn(0x7f22e40b0cf0 msgr2=0x7f22e4159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:03.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.324+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 shutdown_connections 2023-12-04T05:41:03.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.325+0000 7f22eac64640 1 -- 172.21.15.111:0/3161896852 wait complete. 2023-12-04T05:41:03.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:41:03.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.446+0000 7fdea5ada640 1 Processor -- start 2023-12-04T05:41:03.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.446+0000 7fdea5ada640 1 -- start start 2023-12-04T05:41:03.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fdea5ada640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdea013cee0 0x7fdea006c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fdea5ada640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdea0140200 0x7fdea006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fdea5ada640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fdea5ada640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00aea20 con 0x7fdea013cee0 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fdea5ada640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea013d2c0 con 0x7fdea0140200 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fdea5ada640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea015b3a0 con 0x7fdea015a490 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdea013cee0 0x7fdea006c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdea013cee0 0x7fdea006c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57786/0 (socket says 172.21.15.111:57786) 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 -- 172.21.15.111:0/3307879378 learned_addr learned my addr 172.21.15.111:0/3307879378 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:03.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9effd640 1 --2- 172.21.15.111:0/3307879378 >> v2:172.21.15.192:3300/0 conn(0x7fdea0140200 0x7fdea006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9ffff640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 -- 172.21.15.111:0/3307879378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 msgr2=0x7fdea015a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:03.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 -- 172.21.15.111:0/3307879378 >> v2:172.21.15.192:3300/0 conn(0x7fdea0140200 msgr2=0x7fdea006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:03.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/3307879378 >> v2:172.21.15.192:3300/0 conn(0x7fdea0140200 0x7fdea006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9f7fe640 1 -- 172.21.15.111:0/3307879378 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea015ae60 con 0x7fdea013cee0 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.447+0000 7fde9ffff640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea013cee0 0x7fdea006c250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdea0067f50 tx=0x7fde9002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f94afbcb6e9ec46 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fde9e7fc640 1 -- 172.21.15.111:0/3307879378 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900317e0 con 0x7fdea013cee0 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fde9e7fc640 1 -- 172.21.15.111:0/3307879378 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90031940 con 0x7fdea013cee0 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fde9e7fc640 1 -- 172.21.15.111:0/3307879378 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90031cd0 con 0x7fdea013cee0 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 -- 172.21.15.111:0/3307879378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea013cee0 msgr2=0x7fdea006c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea013cee0 0x7fdea006c250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdea0067f50 tx=0x7fde9002b880 comp rx=0 tx=0).stop 2023-12-04T05:41:03.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 -- 172.21.15.111:0/3307879378 shutdown_connections 2023-12-04T05:41:03.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 --2- 172.21.15.111:0/3307879378 >> v2:172.21.15.192:3300/0 conn(0x7fdea0140200 0x7fdea006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 --2- 172.21.15.111:0/3307879378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea013cee0 0x7fdea006c250 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 -- 172.21.15.111:0/3307879378 >> 172.21.15.111:0/3307879378 conn(0x7fdea006a4c0 msgr2=0x7fdea006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:03.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.448+0000 7fdea5ada640 1 -- 172.21.15.111:0/3307879378 shutdown_connections 2023-12-04T05:41:03.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.449+0000 7fdea5ada640 1 -- 172.21.15.111:0/3307879378 wait complete. 2023-12-04T05:41:03.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 Processor -- start 2023-12-04T05:41:03.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 -- start start 2023-12-04T05:41:03.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdea013cee0 0x7fdea01386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 0x7fdea013b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea013c3a0 con 0x7fdea0140200 2023-12-04T05:41:03.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea013c0c0 con 0x7fdea013cee0 2023-12-04T05:41:03.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.450+0000 7fdea5ada640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea013d600 con 0x7fdea015a490 2023-12-04T05:41:03.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 0x7fdea013b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 0x7fdea013b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57788/0 (socket says 172.21.15.111:57788) 2023-12-04T05:41:03.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 -- 172.21.15.111:0/324599744 learned_addr learned my addr 172.21.15.111:0/324599744 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:03.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9ffff640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdea013cee0 0x7fdea01386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 -- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 msgr2=0x7fdea015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 -- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdea013cee0 msgr2=0x7fdea01386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdea013cee0 0x7fdea01386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 -- 172.21.15.111:0/324599744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900094d0 con 0x7fdea0140200 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9ffff640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdea013cee0 0x7fdea01386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde9effd640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 0x7fdea013b6e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fde9400d800 tx=0x7fde9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400b6c0 con 0x7fdea0140200 2023-12-04T05:41:03.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400b820 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea013d8e0 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400bbb0 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.451+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea0138be0 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.452+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea013d460 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.452+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fde9400bd10 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.453+0000 7fde87fff640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde8c06b2a0 0x7fde8c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.453+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde9408a170 con 0x7fdea0140200 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.453+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde8c06b2a0 0x7fde8c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.453+0000 7fde9f7fe640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde8c06b2a0 0x7fde8c06d760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fde90036000 tx=0x7fde9003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:03.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.457+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdea013d460 con 0x7fdea0140200 2023-12-04T05:41:03.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.624+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea013d460 con 0x7fdea0140200 2023-12-04T05:41:03.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.625+0000 7fde87fff640 1 -- 172.21.15.111:0/324599744 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16502 (secure 0 0 0) 0x7fdea013d460 con 0x7fdea0140200 2023-12-04T05:41:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:03.627 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":44,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:41:02.643940+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:41:03.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde8c06b2a0 msgr2=0x7fde8c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde8c06b2a0 0x7fde8c06d760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fde90036000 tx=0x7fde9003d040 comp rx=0 tx=0).stop 2023-12-04T05:41:03.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 msgr2=0x7fdea013b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 0x7fdea013b6e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fde9400d800 tx=0x7fde9400dcc0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 shutdown_connections 2023-12-04T05:41:03.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fde8c06b2a0 0x7fde8c06d760 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdea015a490 0x7fdea015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdea0140200 0x7fdea013b6e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 --2- 172.21.15.111:0/324599744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdea013cee0 0x7fdea01386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.627+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 >> 172.21.15.111:0/324599744 conn(0x7fdea006a4c0 msgr2=0x7fdea006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:03.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.628+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 shutdown_connections 2023-12-04T05:41:03.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.628+0000 7fdea5ada640 1 -- 172.21.15.111:0/324599744 wait complete. 2023-12-04T05:41:03.640 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:41:03.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.738+0000 7f4df78a2640 1 Processor -- start 2023-12-04T05:41:03.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.738+0000 7f4df78a2640 1 -- start start 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df78a2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4df00b20a0 0x7f4df00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df78a2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4df00b2ad0 0x7f4df00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df78a2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00cdcb0 con 0x7f4df00d1780 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df78a2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df00d39e0 con 0x7f4df00b2ad0 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df78a2640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df00d3cc0 con 0x7f4df00b20a0 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df5e3f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4df00b2ad0 0x7f4df00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df5e3f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4df00b2ad0 0x7f4df00b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57798/0 (socket says 172.21.15.111:57798) 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.739+0000 7f4df5e3f640 1 -- 172.21.15.111:0/597821594 learned_addr learned my addr 172.21.15.111:0/597821594 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.740+0000 7f4df4e3d640 1 --2- 172.21.15.111:0/597821594 >> v2:172.21.15.192:3300/0 conn(0x7f4df00b20a0 0x7f4df00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.740+0000 7f4df5e3f640 1 -- 172.21.15.111:0/597821594 >> v2:172.21.15.192:3300/0 conn(0x7f4df00b20a0 msgr2=0x7f4df00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.740+0000 7f4df5e3f640 1 --2- 172.21.15.111:0/597821594 >> v2:172.21.15.192:3300/0 conn(0x7f4df00b20a0 0x7f4df00d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.740+0000 7f4df5e3f640 1 -- 172.21.15.111:0/597821594 >> v1:172.21.15.192:6789/0 conn(0x7f4df00d1780 legacy=0x7f4df00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.740+0000 7f4df5e3f640 1 -- 172.21.15.111:0/597821594 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00cdf50 con 0x7f4df00b2ad0 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.740+0000 7f4df5e3f640 1 --2- 172.21.15.111:0/597821594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b2ad0 0x7f4df00b2f00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4df0162190 tx=0x7f4de802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9de62f5e6ef388b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4de7fff640 1 -- 172.21.15.111:0/597821594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80317e0 con 0x7f4df00b2ad0 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4de7fff640 1 -- 172.21.15.111:0/597821594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8031940 con 0x7f4df00b2ad0 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4de7fff640 1 -- 172.21.15.111:0/597821594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8031cd0 con 0x7f4df00b2ad0 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 -- 172.21.15.111:0/597821594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b2ad0 msgr2=0x7f4df00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 --2- 172.21.15.111:0/597821594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b2ad0 0x7f4df00b2f00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4df0162190 tx=0x7f4de802b880 comp rx=0 tx=0).stop 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 -- 172.21.15.111:0/597821594 shutdown_connections 2023-12-04T05:41:03.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 --2- 172.21.15.111:0/597821594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b2ad0 0x7f4df00b2f00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 --2- 172.21.15.111:0/597821594 >> v2:172.21.15.192:3300/0 conn(0x7f4df00b20a0 0x7f4df00d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 -- 172.21.15.111:0/597821594 >> 172.21.15.111:0/597821594 conn(0x7f4df009c4e0 msgr2=0x7f4df00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:03.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 -- 172.21.15.111:0/597821594 shutdown_connections 2023-12-04T05:41:03.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.741+0000 7f4df78a2640 1 -- 172.21.15.111:0/597821594 wait complete. 2023-12-04T05:41:03.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.742+0000 7f4df78a2640 1 Processor -- start 2023-12-04T05:41:03.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.742+0000 7f4df78a2640 1 -- start start 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df78a2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 0x7f4df01f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df78a2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4df00b2ad0 0x7f4df00b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df78a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df00d1780 0x7f4df00b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df78a2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df014fed0 con 0x7f4df00b20a0 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df78a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00b6e90 con 0x7f4df00d1780 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df78a2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00b7170 con 0x7f4df00b2ad0 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 0x7f4df01f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 0x7f4df01f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57802/0 (socket says 172.21.15.111:57802) 2023-12-04T05:41:03.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 -- 172.21.15.111:0/2618296388 learned_addr learned my addr 172.21.15.111:0/2618296388 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 -- 172.21.15.111:0/2618296388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4df00b2ad0 msgr2=0x7f4df00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4df00b2ad0 0x7f4df00b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 -- 172.21.15.111:0/2618296388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df00d1780 msgr2=0x7f4df00b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df00d1780 0x7f4df00b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.743+0000 7f4df563e640 1 -- 172.21.15.111:0/2618296388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de80094d0 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.744+0000 7f4df563e640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 0x7f4df01f25f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00ba50 tx=0x7f4dec00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.744+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00cbe0 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.744+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00cd80 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.744+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df00b7450 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.744+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec014680 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.744+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00b7a10 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.745+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df00d15f0 con 0x7f4df00b20a0 2023-12-04T05:41:03.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.745+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f4dec0147e0 con 0x7f4df00b20a0 2023-12-04T05:41:03.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.745+0000 7f4de5ffb640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4dd406b2a0 0x7f4dd406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:03.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.746+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dec089a50 con 0x7f4df00b20a0 2023-12-04T05:41:03.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.746+0000 7f4df4e3d640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4dd406b2a0 0x7f4dd406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:03.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.746+0000 7f4df4e3d640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4dd406b2a0 0x7f4dd406d760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4df0159fb0 tx=0x7f4de0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:03.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.750+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dec054970 con 0x7f4df00b20a0 2023-12-04T05:41:03.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.916+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4df00693d0 con 0x7f4df00b20a0 2023-12-04T05:41:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.917+0000 7f4de5ffb640 1 -- 172.21.15.111:0/2618296388 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16502 (secure 0 0 0) 0x7f4dec019090 con 0x7f4df00b20a0 2023-12-04T05:41:03.919 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:03.919 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":44,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:41:02.643940+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:41:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4dd406b2a0 msgr2=0x7f4dd406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4dd406b2a0 0x7f4dd406d760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4df0159fb0 tx=0x7f4de0008040 comp rx=0 tx=0).stop 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 msgr2=0x7f4df01f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 0x7f4df01f25f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00ba50 tx=0x7f4dec00bf10 comp rx=0 tx=0).stop 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 shutdown_connections 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f4dd406b2a0 0x7f4dd406d760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4df00d1780 0x7f4df00b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4df00b2ad0 0x7f4df00b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 --2- 172.21.15.111:0/2618296388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4df00b20a0 0x7f4df01f25f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 >> 172.21.15.111:0/2618296388 conn(0x7f4df009c4e0 msgr2=0x7f4df006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 shutdown_connections 2023-12-04T05:41:03.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:03.919+0000 7f4df78a2640 1 -- 172.21.15.111:0/2618296388 wait complete. 2023-12-04T05:41:03.931 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-04T05:41:04.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.033+0000 7f1dcf7fb640 1 Processor -- start 2023-12-04T05:41:04.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.033+0000 7f1dcf7fb640 1 -- start start 2023-12-04T05:41:04.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcf7fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 0x7f1dc815e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcf7fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1dc80b37e0 0x7f1dc80b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcf7fb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc80b3f80 con 0x7f1dc8070880 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcf7fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80b4540 con 0x7f1dc80b37e0 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcf7fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b2430 con 0x7f1dc80cec00 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcdd98640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1dc8070880 0x7f1dc8070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39576/0 (socket says 172.21.15.111:39576) 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcdd98640 1 -- 172.21.15.111:0/2428550725 learned_addr learned my addr 172.21.15.111:0/2428550725 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869642920 0 0) 0x7f1dc80b3f80 con 0x7f1dc8070880 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db4003520 con 0x7f1dc8070880 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dccd96640 1 --2- 172.21.15.111:0/2428550725 >> v2:172.21.15.192:3300/0 conn(0x7f1dc80b37e0 0x7f1dc80b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.034+0000 7f1dcd597640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 0x7f1dc815e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683069491 0 0) 0x7f1db4003520 con 0x7f1dc8070880 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 msgr2=0x7f1dc815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:04.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 0x7f1dc815e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 >> v2:172.21.15.192:3300/0 conn(0x7f1dc80b37e0 msgr2=0x7f1dc80b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 --2- 172.21.15.111:0/2428550725 >> v2:172.21.15.192:3300/0 conn(0x7f1dc80b37e0 0x7f1dc80b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc80b4260 con 0x7f1dc8070880 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f1dc0001670 con 0x7f1dc8070880 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 >> v1:172.21.15.111:6789/0 conn(0x7f1dc8070880 legacy=0x7f1dc8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 0x7f1db4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1db4003ef0 0x7f1db40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db4004d00 0x7f1db4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b2430 con 0x7f1db4004d00 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db400b980 con 0x7f1db4003ef0 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db400bb10 con 0x7f1dc80cec00 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcdd98640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db4004d00 0x7f1db4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcd597640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 0x7f1db4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dccd96640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1db4003ef0 0x7f1db40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcdd98640 1 -- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 msgr2=0x7f1db4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcdd98640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc80cec00 0x7f1db4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcdd98640 1 -- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1db4003ef0 msgr2=0x7f1db40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcdd98640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1db4003ef0 0x7f1db40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.035+0000 7f1dcdd98640 1 -- 172.21.15.111:0/2428550725 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4003520 con 0x7f1db4004d00 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.036+0000 7f1dcdd98640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db4004d00 0x7f1db4005180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000c2c0 tx=0x7f1db40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fb9eb5d0f36d46 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.036+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0005ad0 con 0x7f1db4004d00 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.036+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc0005c30 con 0x7f1db4004d00 2023-12-04T05:41:04.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.036+0000 7f1dbffff640 1 -- 172.21.15.111:0/2428550725 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc002c8b0 con 0x7f1db4004d00 2023-12-04T05:41:04.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/2428550725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db4004d00 msgr2=0x7f1db4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/2428550725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db4004d00 0x7f1db4005180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000c2c0 tx=0x7f1db40057f0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcdd98640 1 -- 172.21.15.111:0/2428550725 reap_dead start 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/2428550725 shutdown_connections 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/2428550725 >> 172.21.15.111:0/2428550725 conn(0x7f1dc806a4c0 msgr2=0x7f1dc806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/2428550725 shutdown_connections 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.037+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/2428550725 wait complete. 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.038+0000 7f1dcf7fb640 1 Processor -- start 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.038+0000 7f1dcf7fb640 1 -- start start 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 0x7f1dc8143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc815f390 0x7f1dc8144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc8151ad0 0x7f1dc8151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8153cd0 con 0x7f1dc815efb0 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8153f70 con 0x7f1dc8151ad0 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8154250 con 0x7f1dc815f390 2023-12-04T05:41:04.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 0x7f1dc8143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 0x7f1dc8143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57824/0 (socket says 172.21.15.111:57824) 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 -- 172.21.15.111:0/970946142 learned_addr learned my addr 172.21.15.111:0/970946142 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcdd98640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc8151ad0 0x7f1dc8151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 -- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc815f390 msgr2=0x7f1dc8144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dccd96640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc815f390 0x7f1dc8144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc815f390 0x7f1dc8144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 -- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc8151ad0 msgr2=0x7f1dc8151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc8151ad0 0x7f1dc8151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 -- 172.21.15.111:0/970946142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc8154530 con 0x7f1dc815efb0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcd597640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 0x7f1dc8143bd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4011e30 tx=0x7f1dc400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400e040 con 0x7f1dc815efb0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc400f040 con 0x7f1dc815efb0 2023-12-04T05:41:04.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8154810 con 0x7f1dc815efb0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.039+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4009c00 con 0x7f1dc815efb0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.040+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc815c570 con 0x7f1dc815efb0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.040+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc80b3300 con 0x7f1dc815efb0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.041+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1dc4015850 con 0x7f1dc815efb0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.041+0000 7f1dbdffb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1dac06b2a0 0x7f1dac06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.041+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dc408ab10 con 0x7f1dc815efb0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.041+0000 7f1dccd96640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1dac06b2a0 0x7f1dac06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.042+0000 7f1dccd96640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1dac06b2a0 0x7f1dac06d760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80ae9a0 tx=0x7f1db8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:04.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.046+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1dc40558e0 con 0x7f1dc815efb0 2023-12-04T05:41:04.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.247+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1dc80693d0 con 0x7f1dc815efb0 2023-12-04T05:41:04.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.309+0000 7f1dbdffb640 1 -- 172.21.15.111:0/970946142 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1dc4058f00 con 0x7f1dc815efb0 2023-12-04T05:41:04.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1dac06b2a0 msgr2=0x7f1dac06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1dac06b2a0 0x7f1dac06d760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80ae9a0 tx=0x7f1db8008040 comp rx=0 tx=0).stop 2023-12-04T05:41:04.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 msgr2=0x7f1dc8143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 0x7f1dc8143bd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4011e30 tx=0x7f1dc400f580 comp rx=0 tx=0).stop 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 shutdown_connections 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1dac06b2a0 0x7f1dac06d760 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc8151ad0 0x7f1dc8151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc815f390 0x7f1dc8144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 --2- 172.21.15.111:0/970946142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1dc815efb0 0x7f1dc8143bd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 >> 172.21.15.111:0/970946142 conn(0x7f1dc806a4c0 msgr2=0x7f1dc806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 shutdown_connections 2023-12-04T05:41:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.311+0000 7f1dcf7fb640 1 -- 172.21.15.111:0/970946142 wait complete. 2023-12-04T05:41:04.324 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-04T05:41:04.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-04T05:41:04.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.424+0000 7fe4193c9640 1 Processor -- start 2023-12-04T05:41:04.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.424+0000 7fe4193c9640 1 -- start start 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe4193c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 0x7fe4140d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe4193c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 0x7fe414160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe4193c9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140d1b30 con 0x7fe414161530 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe4193c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41415da00 con 0x7fe414146a60 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe4193c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe414161d60 con 0x7fe4140d16d0 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe412ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 0x7fe4140d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 0x7fe414160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe412ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 0x7fe4140d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44208/0 (socket says 172.21.15.111:44208) 2023-12-04T05:41:04.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 0x7fe414160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:36148/0 (socket says 172.21.15.111:36148) 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe4137fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fe414161530 0x7fe4140b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:46200/0 (socket says 172.21.15.111:46200) 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.425+0000 7fe412ffd640 1 -- 172.21.15.111:0/3134069372 learned_addr learned my addr 172.21.15.111:0/3134069372 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.426+0000 7fe40bfff640 1 -- 172.21.15.111:0/3134069372 >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 msgr2=0x7fe4140d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.426+0000 7fe40bfff640 1 --2- 172.21.15.111:0/3134069372 >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 0x7fe4140d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.426+0000 7fe40bfff640 1 -- 172.21.15.111:0/3134069372 >> v1:172.21.15.192:6789/0 conn(0x7fe414161530 legacy=0x7fe4140b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.426+0000 7fe40bfff640 1 -- 172.21.15.111:0/3134069372 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41415dcc0 con 0x7fe4140d16d0 2023-12-04T05:41:04.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.426+0000 7fe412ffd640 1 --2- 172.21.15.111:0/3134069372 >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 0x7fe4140d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:41:04.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.426+0000 7fe40bfff640 1 --2- 172.21.15.111:0/3134069372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 0x7fe414160ff0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0097a0 tx=0x7fe3fc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c68cbf1a6a7d2950 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:04.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4127fc640 1 -- 172.21.15.111:0/3134069372 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc02e070 con 0x7fe4140d16d0 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4127fc640 1 -- 172.21.15.111:0/3134069372 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0315a0 con 0x7fe4140d16d0 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4127fc640 1 -- 172.21.15.111:0/3134069372 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc031930 con 0x7fe4140d16d0 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 -- 172.21.15.111:0/3134069372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 msgr2=0x7fe414160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 --2- 172.21.15.111:0/3134069372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 0x7fe414160ff0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0097a0 tx=0x7fe3fc02b840 comp rx=0 tx=0).stop 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 -- 172.21.15.111:0/3134069372 shutdown_connections 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 --2- 172.21.15.111:0/3134069372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140d16d0 0x7fe414160ff0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 --2- 172.21.15.111:0/3134069372 >> v2:172.21.15.192:3300/0 conn(0x7fe414146a60 0x7fe4140d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 -- 172.21.15.111:0/3134069372 >> 172.21.15.111:0/3134069372 conn(0x7fe41406a6a0 msgr2=0x7fe41406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 -- 172.21.15.111:0/3134069372 shutdown_connections 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.427+0000 7fe4193c9640 1 -- 172.21.15.111:0/3134069372 wait complete. 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.428+0000 7fe4193c9640 1 Processor -- start 2023-12-04T05:41:04.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.428+0000 7fe4193c9640 1 -- start start 2023-12-04T05:41:04.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4193c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4140d16d0 0x7fe4140aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4193c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe414146a60 0x7fe4140a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4193c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 0x7fe4140cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4193c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140a9850 con 0x7fe414161530 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4193c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140a89a0 con 0x7fe4140d16d0 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4193c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140a8c40 con 0x7fe414146a60 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 0x7fe4140cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 0x7fe4140cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57836/0 (socket says 172.21.15.111:57836) 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 -- 172.21.15.111:0/1217199901 learned_addr learned my addr 172.21.15.111:0/1217199901 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe40bfff640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe414146a60 0x7fe4140a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe412ffd640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4140d16d0 0x7fe4140aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 -- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe414146a60 msgr2=0x7fe4140a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe414146a60 0x7fe4140a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 -- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4140d16d0 msgr2=0x7fe4140aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:04.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4140d16d0 0x7fe4140aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.429+0000 7fe4137fe640 1 -- 172.21.15.111:0/1217199901 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040094d0 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.430+0000 7fe4137fe640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 0x7fe4140cf150 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c730 tx=0x7fe40c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.430+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00ce40 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.430+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc009400 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.430+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00a640 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.430+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00a9f0 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.430+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4140a9200 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.431+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4140a99c0 con 0x7fe414161530 2023-12-04T05:41:04.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.431+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe40c00ab50 con 0x7fe414161530 2023-12-04T05:41:04.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.432+0000 7fe40b7fe640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe3f406b2a0 0x7fe3f406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:04.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.432+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe40c089320 con 0x7fe414161530 2023-12-04T05:41:04.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.432+0000 7fe412ffd640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe3f406b2a0 0x7fe3f406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:04.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.432+0000 7fe412ffd640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe3f406b2a0 0x7fe3f406d760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe404006ca0 tx=0x7fe404009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:04.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.436+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe40c0541c0 con 0x7fe414161530 2023-12-04T05:41:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:04.637+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe4140693d0 con 0x7fe414161530 2023-12-04T05:41:05.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.320+0000 7fe40b7fe640 1 -- 172.21.15.111:0/1217199901 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe40c0577e0 con 0x7fe414161530 2023-12-04T05:41:05.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe3f406b2a0 msgr2=0x7fe3f406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe3f406b2a0 0x7fe3f406d760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe404006ca0 tx=0x7fe404009240 comp rx=0 tx=0).stop 2023-12-04T05:41:05.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 msgr2=0x7fe4140cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 0x7fe4140cf150 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c730 tx=0x7fe40c00cbf0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 shutdown_connections 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe3f406b2a0 0x7fe3f406d760 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe414161530 0x7fe4140cf150 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe414146a60 0x7fe4140a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.322+0000 7fe4193c9640 1 --2- 172.21.15.111:0/1217199901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4140d16d0 0x7fe4140aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.323+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 >> 172.21.15.111:0/1217199901 conn(0x7fe41406a6a0 msgr2=0x7fe41406aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.323+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 shutdown_connections 2023-12-04T05:41:05.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.323+0000 7fe4193c9640 1 -- 172.21.15.111:0/1217199901 wait complete. 2023-12-04T05:41:05.337 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-04T05:41:05.337 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:41:05.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.441+0000 7fad89eee640 1 Processor -- start 2023-12-04T05:41:05.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.441+0000 7fad89eee640 1 -- start start 2023-12-04T05:41:05.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad89eee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad89eee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 0x7fad840d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad89eee640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840cdcb0 con 0x7fad840b2ad0 2023-12-04T05:41:05.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad89eee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad840d20a0 con 0x7fad840b20a0 2023-12-04T05:41:05.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad89eee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840d39e0 con 0x7fad840d1780 2023-12-04T05:41:05.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad82ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 0x7fad840d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad837fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad82ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 0x7fad840d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44230/0 (socket says 172.21.15.111:44230) 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad82ffd640 1 -- 172.21.15.111:0/509923589 learned_addr learned my addr 172.21.15.111:0/509923589 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad837fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:36168/0 (socket says 172.21.15.111:36168) 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.442+0000 7fad83fff640 1 --1- 172.21.15.111:0/509923589 >> v1:172.21.15.192:6789/0 conn(0x7fad840b2ad0 0x7fad840ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:46204/0 (socket says 172.21.15.111:46204) 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.443+0000 7fad837fe640 1 -- 172.21.15.111:0/509923589 >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 msgr2=0x7fad840d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.443+0000 7fad837fe640 1 --2- 172.21.15.111:0/509923589 >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 0x7fad840d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.443+0000 7fad837fe640 1 -- 172.21.15.111:0/509923589 >> v1:172.21.15.192:6789/0 conn(0x7fad840b2ad0 legacy=0x7fad840ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.443+0000 7fad837fe640 1 -- 172.21.15.111:0/509923589 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840d3cc0 con 0x7fad840d1780 2023-12-04T05:41:05.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.443+0000 7fad82ffd640 1 --2- 172.21.15.111:0/509923589 >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 0x7fad840d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:41:05.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.443+0000 7fad837fe640 1 --2- 172.21.15.111:0/509923589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840d01e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad740097a0 tx=0x7fad7402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c723cdd4e31876 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:05.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad827fc640 1 -- 172.21.15.111:0/509923589 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7402e070 con 0x7fad840d1780 2023-12-04T05:41:05.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad827fc640 1 -- 172.21.15.111:0/509923589 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740315a0 con 0x7fad840d1780 2023-12-04T05:41:05.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad827fc640 1 -- 172.21.15.111:0/509923589 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74031930 con 0x7fad840d1780 2023-12-04T05:41:05.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 -- 172.21.15.111:0/509923589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 msgr2=0x7fad840d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 --2- 172.21.15.111:0/509923589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840d01e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad740097a0 tx=0x7fad7402b840 comp rx=0 tx=0).stop 2023-12-04T05:41:05.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 -- 172.21.15.111:0/509923589 shutdown_connections 2023-12-04T05:41:05.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 --2- 172.21.15.111:0/509923589 >> v2:172.21.15.192:3300/0 conn(0x7fad840b20a0 0x7fad840d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 --2- 172.21.15.111:0/509923589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840d01e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 -- 172.21.15.111:0/509923589 >> 172.21.15.111:0/509923589 conn(0x7fad8409c4e0 msgr2=0x7fad840d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:05.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 -- 172.21.15.111:0/509923589 shutdown_connections 2023-12-04T05:41:05.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.444+0000 7fad89eee640 1 -- 172.21.15.111:0/509923589 wait complete. 2023-12-04T05:41:05.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.445+0000 7fad89eee640 1 Processor -- start 2023-12-04T05:41:05.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 -- start start 2023-12-04T05:41:05.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad840b20a0 0x7fad84159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 0x7fad84160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840cd1f0 con 0x7fad840b2ad0 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8415ff10 con 0x7fad840b20a0 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad89eee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840b5f60 con 0x7fad840d1780 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad82ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 0x7fad84160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad82ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 0x7fad84160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57848/0 (socket says 172.21.15.111:57848) 2023-12-04T05:41:05.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.446+0000 7fad82ffd640 1 -- 172.21.15.111:0/439217726 learned_addr learned my addr 172.21.15.111:0/439217726 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad837fe640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad840b20a0 0x7fad84159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad83fff640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad82ffd640 1 -- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 msgr2=0x7fad840cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad82ffd640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad82ffd640 1 -- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad840b20a0 msgr2=0x7fad84159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad82ffd640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad840b20a0 0x7fad84159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad82ffd640 1 -- 172.21.15.111:0/439217726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780094d0 con 0x7fad840b2ad0 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad82ffd640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 0x7fad84160310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fad7800e9e0 tx=0x7fad7800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800c9d0 con 0x7fad840b2ad0 2023-12-04T05:41:05.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7800cb30 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800cea0 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad74009400 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.447+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8415f950 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.448+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840693d0 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.448+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fad78013470 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.449+0000 7fad6bfff640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad7006b2a0 0x7fad7006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.449+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad78089e70 con 0x7fad840b2ad0 2023-12-04T05:41:05.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.449+0000 7fad837fe640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad7006b2a0 0x7fad7006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.449+0000 7fad837fe640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad7006b2a0 0x7fad7006d760 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad7403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:05.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.453+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad78054ce0 con 0x7fad840b2ad0 2023-12-04T05:41:05.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.657+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad840b60d0 con 0x7fad840b2ad0 2023-12-04T05:41:05.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.658+0000 7fad6bfff640 1 -- 172.21.15.111:0/439217726 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fad840b60d0 con 0x7fad840b2ad0 2023-12-04T05:41:05.660 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/956157809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":956157809},{"type":"v1","addr":"172.21.15.192:6835","nonce":956157809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/619465105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":619465105},{"type":"v1","addr":"172.21.15.111:6836","nonce":619465105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:41:04.639249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:41:05.660 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 20 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad7006b2a0 msgr2=0x7fad7006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad7006b2a0 0x7fad7006d760 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad7403d040 comp rx=0 tx=0).stop 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 msgr2=0x7fad84160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 0x7fad84160310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fad7800e9e0 tx=0x7fad7800eea0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 shutdown_connections 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fad7006b2a0 0x7fad7006d760 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad840d1780 0x7fad840cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad840b2ad0 0x7fad84160310 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 --2- 172.21.15.111:0/439217726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad840b20a0 0x7fad84159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 >> 172.21.15.111:0/439217726 conn(0x7fad8409c4e0 msgr2=0x7fad8406b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:05.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 shutdown_connections 2023-12-04T05:41:05.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.660+0000 7fad89eee640 1 -- 172.21.15.111:0/439217726 wait complete. 2023-12-04T05:41:05.672 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:41:05.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.770+0000 7f566047c640 1 Processor -- start 2023-12-04T05:41:05.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.770+0000 7f566047c640 1 -- start start 2023-12-04T05:41:05.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f566047c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f565815aba0 0x7f5658164e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f566047c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56580aea20 con 0x7f565806c960 2023-12-04T05:41:05.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f566047c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565813d2c0 con 0x7f565813cee0 2023-12-04T05:41:05.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f566047c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565815b3a0 con 0x7f565815aba0 2023-12-04T05:41:05.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f565da17640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f565806c960 0x7f565815a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39588/0 (socket says 172.21.15.111:39588) 2023-12-04T05:41:05.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f565da17640 1 -- 172.21.15.111:0/3707940007 learned_addr learned my addr 172.21.15.111:0/3707940007 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f565ea19640 1 --2- 172.21.15.111:0/3707940007 >> v2:172.21.15.192:3300/0 conn(0x7f565815aba0 0x7f5658164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903567948 0 0) 0x7f56580aea20 con 0x7f565806c960 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.771+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5640003520 con 0x7f565806c960 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2385856644 0 0) 0x7f5640003520 con 0x7f565806c960 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 >> v2:172.21.15.192:3300/0 conn(0x7f565815aba0 msgr2=0x7f5658164e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 --2- 172.21.15.111:0/3707940007 >> v2:172.21.15.192:3300/0 conn(0x7f565815aba0 0x7f5658164e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 >> v1:172.21.15.192:6789/0 conn(0x7f565813cee0 legacy=0x7f565806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565815b0c0 con 0x7f565806c960 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f564c001670 con 0x7f565806c960 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 >> v1:172.21.15.111:6789/0 conn(0x7f565806c960 legacy=0x7f565815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5640003970 0x7f5640003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5640004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5640005180 0x7f564000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565815b3a0 con 0x7f56400047c0 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564000e330 con 0x7f5640003970 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564000e4f0 con 0x7f5640005180 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565da17640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5640004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565e218640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5640003970 0x7f5640003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565ea19640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5640005180 0x7f564000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565da17640 1 -- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5640005180 msgr2=0x7f564000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565da17640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5640005180 0x7f564000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565da17640 1 -- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5640003970 msgr2=0x7f5640003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565da17640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5640003970 0x7f5640003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565da17640 1 -- 172.21.15.111:0/3707940007 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5640003520 con 0x7f56400047c0 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565ea19640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5640005180 0x7f564000db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.772+0000 7f565ea19640 1 -- 172.21.15.111:0/3707940007 reap_dead start 2023-12-04T05:41:05.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f565da17640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5640004c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c240 tx=0x7f56400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fbcde16d27c28 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c005ad0 con 0x7f56400047c0 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c005c30 con 0x7f56400047c0 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f565d216640 1 -- 172.21.15.111:0/3707940007 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c02ca90 con 0x7f56400047c0 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 -- 172.21.15.111:0/3707940007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 msgr2=0x7f5640004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5640004c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c240 tx=0x7f56400057f0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 -- 172.21.15.111:0/3707940007 shutdown_connections 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 --2- 172.21.15.111:0/3707940007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5640004c40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 -- 172.21.15.111:0/3707940007 >> 172.21.15.111:0/3707940007 conn(0x7f565806a4c0 msgr2=0x7f565806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 -- 172.21.15.111:0/3707940007 shutdown_connections 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.773+0000 7f566047c640 1 -- 172.21.15.111:0/3707940007 wait complete. 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 Processor -- start 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 -- start start 2023-12-04T05:41:05.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5658139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5658138850 0x7f56580b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580b0850 0x7f56581eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580af270 con 0x7f56400047c0 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580af550 con 0x7f5658138850 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f566047c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56580af830 con 0x7f56580b0850 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f565e218640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5658139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f565e218640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5658139b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57866/0 (socket says 172.21.15.111:57866) 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.775+0000 7f565e218640 1 -- 172.21.15.111:0/2390225627 learned_addr learned my addr 172.21.15.111:0/2390225627 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565da17640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5658138850 0x7f56580b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565ea19640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580b0850 0x7f56581eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565e218640 1 -- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580b0850 msgr2=0x7f56581eeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565e218640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580b0850 0x7f56581eeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565e218640 1 -- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5658138850 msgr2=0x7f56580b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565e218640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5658138850 0x7f56580b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565e218640 1 -- 172.21.15.111:0/2390225627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56580afb10 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565da17640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5658138850 0x7f56580b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f565e218640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5658139b00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f565000f5f0 tx=0x7f565000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500052e0 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5650005440 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56580afdf0 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650003810 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.776+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565813a040 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.777+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56580693d0 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.777+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f5650003970 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.778+0000 7f5656ffd640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f564406b2a0 0x7f564406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.778+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5650089920 con 0x7f56400047c0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.778+0000 7f565da17640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f564406b2a0 0x7f564406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:05.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.778+0000 7f565da17640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f564406b2a0 0x7f564406d760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f564c034000 tx=0x7f564c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:05.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.782+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5650054790 con 0x7f56400047c0 2023-12-04T05:41:05.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.949+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f565813e440 con 0x7f56400047c0 2023-12-04T05:41:05.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.950+0000 7f5656ffd640 1 -- 172.21.15.111:0/2390225627 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16489 (secure 0 0 0) 0x7f565813e440 con 0x7f56400047c0 2023-12-04T05:41:05.952 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:05.952 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":45,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:41:04.299927+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:41:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f564406b2a0 msgr2=0x7f564406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f564406b2a0 0x7f564406d760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f564c034000 tx=0x7f564c009030 comp rx=0 tx=0).stop 2023-12-04T05:41:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 msgr2=0x7f5658139b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5658139b00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f565000f5f0 tx=0x7f565000fab0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 shutdown_connections 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f564406b2a0 0x7f564406d760 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580b0850 0x7f56581eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5658138850 0x7f56580b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 --2- 172.21.15.111:0/2390225627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56400047c0 0x7f5658139b00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 >> 172.21.15.111:0/2390225627 conn(0x7f565806a4c0 msgr2=0x7f565813e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 shutdown_connections 2023-12-04T05:41:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:05.952+0000 7f566047c640 1 -- 172.21.15.111:0/2390225627 wait complete. 2023-12-04T05:41:05.966 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-04T05:41:05.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:41:06.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.064+0000 7fee6f968640 1 Processor -- start 2023-12-04T05:41:06.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.064+0000 7fee6f968640 1 -- start start 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6f968640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fee6815dab0 0x7fee680703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6f968640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee68154220 con 0x7fee680b3310 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6f968640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6815d2f0 con 0x7fee6806cae0 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6f968640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6815a3c0 con 0x7fee6815dab0 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6cf03640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fee680b3310 0x7fee68159cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39600/0 (socket says 172.21.15.111:39600) 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6cf03640 1 -- 172.21.15.111:0/1273261692 learned_addr learned my addr 172.21.15.111:0/1273261692 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:06.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257650455 0 0) 0x7fee58001110 con 0x7fee680b3310 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee6df05640 1 --2- 172.21.15.111:0/1273261692 >> v2:172.21.15.192:3300/0 conn(0x7fee6815dab0 0x7fee680703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee54004630 con 0x7fee680b3310 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562109483 0 0) 0x7fee54004630 con 0x7fee680b3310 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 >> v2:172.21.15.192:3300/0 conn(0x7fee6815dab0 msgr2=0x7fee680703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 --2- 172.21.15.111:0/1273261692 >> v2:172.21.15.192:3300/0 conn(0x7fee6815dab0 0x7fee680703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 >> v1:172.21.15.192:6789/0 conn(0x7fee6806cae0 legacy=0x7fee6806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6815a5c0 con 0x7fee680b3310 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fee58001800 con 0x7fee680b3310 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.065+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 >> v1:172.21.15.111:6789/0 conn(0x7fee680b3310 legacy=0x7fee68159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee5ffff640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54004a80 0x7fee54004ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee5ffff640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee540058d0 0x7fee54005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee5ffff640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee54006290 0x7fee5400ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815a3c0 con 0x7fee54006290 2023-12-04T05:41:06.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400d130 con 0x7fee540058d0 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400d2f0 con 0x7fee54004a80 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6df05640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee54006290 0x7fee5400ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6cf03640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee540058d0 0x7fee54005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6d704640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54004a80 0x7fee54004ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6df05640 1 -- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54004a80 msgr2=0x7fee54004ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6df05640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54004a80 0x7fee54004ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6df05640 1 -- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee540058d0 msgr2=0x7fee54005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6df05640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee540058d0 0x7fee54005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6df05640 1 -- 172.21.15.111:0/1273261692 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54004630 con 0x7fee54006290 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.066+0000 7fee6d704640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54004a80 0x7fee54004ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6df05640 1 -- 172.21.15.111:0/1273261692 reap_dead start 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6df05640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee54006290 0x7fee5400ca90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee60009fc0 tx=0x7fee54006900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b46dc2cc5271a0d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6002cc60 con 0x7fee54006290 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6000f040 con 0x7fee54006290 2023-12-04T05:41:06.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee5ffff640 1 -- 172.21.15.111:0/1273261692 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60003990 con 0x7fee54006290 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 -- 172.21.15.111:0/1273261692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee54006290 msgr2=0x7fee5400ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee54006290 0x7fee5400ca90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee60009fc0 tx=0x7fee54006900 comp rx=0 tx=0).stop 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 -- 172.21.15.111:0/1273261692 shutdown_connections 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 --2- 172.21.15.111:0/1273261692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee54006290 0x7fee5400ca90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 -- 172.21.15.111:0/1273261692 >> 172.21.15.111:0/1273261692 conn(0x7fee6806a4c0 msgr2=0x7fee6806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 -- 172.21.15.111:0/1273261692 shutdown_connections 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.067+0000 7fee6f968640 1 -- 172.21.15.111:0/1273261692 wait complete. 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 Processor -- start 2023-12-04T05:41:06.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 -- start start 2023-12-04T05:41:06.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54006290 0x7fee68151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee680a8460 0x7fee680a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 0x7fee680708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68070de0 con 0x7fee680703f0 2023-12-04T05:41:06.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681524c0 con 0x7fee680a8460 2023-12-04T05:41:06.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6f968640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee680a9b10 con 0x7fee54006290 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6df05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 0x7fee680708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6df05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 0x7fee680708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57886/0 (socket says 172.21.15.111:57886) 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.069+0000 7fee6df05640 1 -- 172.21.15.111:0/3631057619 learned_addr learned my addr 172.21.15.111:0/3631057619 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6cf03640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee680a8460 0x7fee680a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6d704640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54006290 0x7fee68151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6df05640 1 -- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54006290 msgr2=0x7fee68151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6df05640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54006290 0x7fee68151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6df05640 1 -- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee680a8460 msgr2=0x7fee680a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6df05640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee680a8460 0x7fee680a88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6df05640 1 -- 172.21.15.111:0/3631057619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee680a8f00 con 0x7fee680703f0 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6cf03640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee680a8460 0x7fee680a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6df05640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 0x7fee680708a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fee6000e200 tx=0x7fee60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60003da0 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee60035a00 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee680a91e0 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60035db0 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.070+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee680a97a0 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.071+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee680a9960 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.072+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fee60004030 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.072+0000 7fee5dffb640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee4c06b2d0 0x7fee4c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.072+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee600ac810 con 0x7fee680703f0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.072+0000 7fee6d704640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee4c06b2d0 0x7fee4c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.073+0000 7fee6d704640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee4c06b2d0 0x7fee4c06d790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fee64000c00 tx=0x7fee64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.076+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee60077830 con 0x7fee680703f0 2023-12-04T05:41:06.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.240+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee680cb200 con 0x7fee4c06b2d0 2023-12-04T05:41:06.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.247+0000 7fee5dffb640 1 -- 172.21.15.111:0/3631057619 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275196 (secure 0 0 0) 0x7fee680cb200 con 0x7fee4c06b2d0 2023-12-04T05:41:06.253 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:41:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee4c06b2d0 msgr2=0x7fee4c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee4c06b2d0 0x7fee4c06d790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fee64000c00 tx=0x7fee64009040 comp rx=0 tx=0).stop 2023-12-04T05:41:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 msgr2=0x7fee680708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 0x7fee680708a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fee6000e200 tx=0x7fee60009750 comp rx=0 tx=0).stop 2023-12-04T05:41:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 shutdown_connections 2023-12-04T05:41:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee4c06b2d0 0x7fee4c06d790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee680703f0 0x7fee680708a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee680a8460 0x7fee680a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 --2- 172.21.15.111:0/3631057619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee54006290 0x7fee68151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 >> 172.21.15.111:0/3631057619 conn(0x7fee6806a4c0 msgr2=0x7fee680cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:06.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.250+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 shutdown_connections 2023-12-04T05:41:06.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.251+0000 7fee6f968640 1 -- 172.21.15.111:0/3631057619 wait complete. 2023-12-04T05:41:06.263 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-12-04T05:41:04.493400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220984,"kb_used_data":4772,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753736,"statfs":{"total":773094113280,"available":772867825664,"internally_reserved":0,"allocated":4886528,"data_stored":2643731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000563"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:02.292103+0000","last_change":"2023-12-04T05:41:02.292103+0000","last_active":"2023-12-04T05:41:02.292103+0000","last_peered":"2023-12-04T05:41:02.292103+0000","last_clean":"2023-12-04T05:41:02.292103+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:02.292103+0000","last_undegraded":"2023-12-04T05:41:02.292103+0000","last_fullsized":"2023-12-04T05:41:02.292103+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-04T05:41:01.288023+0000","last_change":"2023-12-04T05:41:01.288023+0000","last_active":"2023-12-04T05:41:01.288023+0000","last_peered":"2023-12-04T05:41:01.288023+0000","last_clean":"2023-12-04T05:41:01.288023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-04T05:41:01.288023+0000","last_undegraded":"2023-12-04T05:41:01.288023+0000","last_fullsized":"2023-12-04T05:41:01.288023+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-12-04T05:40:58.912962+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:40:58.912962+0000","last_peered":"2023-12-04T05:40:58.912962+0000","last_clean":"2023-12-04T05:40:58.912962+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:40:58.912962+0000","last_undegraded":"2023-12-04T05:40:58.912962+0000","last_fullsized":"2023-12-04T05:40:58.912962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":780300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.435},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.505},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.57699999999999996}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65400000000000003}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":466944,"data_stored":184805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.627}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.54300000000000004}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.498},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.60999999999999999}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":184805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.628}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":184805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39600000000000002}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.625}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.65500000000000003}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.67800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":184805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.25},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.36099999999999999}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.28999999999999998}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.437}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46800000000000003}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":184805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.253}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66600000000000004}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.627}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.495}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":184805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.255},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.29599999999999999}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.627},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.749}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.76400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":754601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.313}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.33700000000000002}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30299999999999999}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44600000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.436},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48899999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.39600000000000002}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:41:06.268 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:41:06.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.402+0000 7fef2a451640 1 Processor -- start 2023-12-04T05:41:06.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.402+0000 7fef2a451640 1 -- start start 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.402+0000 7fef2a451640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef24071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef2a451640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fef24071710 0x7fef2406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef2a451640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef241405a0 con 0x7fef2406c790 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef2a451640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2406cc00 con 0x7fef24071710 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef2a451640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef24140880 con 0x7fef241401c0 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef289ee640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fef2406c790 0x7fef240703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39612/0 (socket says 172.21.15.111:39612) 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef289ee640 1 -- 172.21.15.111:0/2492950066 learned_addr learned my addr 172.21.15.111:0/2492950066 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef237fe640 1 --2- 172.21.15.111:0/2492950066 >> v2:172.21.15.192:3300/0 conn(0x7fef24071710 0x7fef2406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef23fff640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef24071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810504343 0 0) 0x7fef241405a0 con 0x7fef2406c790 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef0c003520 con 0x7fef2406c790 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008379897 0 0) 0x7fef0c003520 con 0x7fef2406c790 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 msgr2=0x7fef24071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef24071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 >> v2:172.21.15.192:3300/0 conn(0x7fef24071710 msgr2=0x7fef2406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 --2- 172.21.15.111:0/2492950066 >> v2:172.21.15.192:3300/0 conn(0x7fef24071710 0x7fef2406c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2415db00 con 0x7fef2406c790 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fef1c0019f0 con 0x7fef2406c790 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef23fff640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef24071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 >> v1:172.21.15.111:6789/0 conn(0x7fef2406c790 legacy=0x7fef240703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0c003970 0x7fef0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef0c004d00 0x7fef0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2406cc00 con 0x7fef0c003970 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef24140880 con 0x7fef0c004d00 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c00b980 con 0x7fef241401c0 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef237fe640 1 --2- 172.21.15.111:0/2492950066 >> v2:172.21.15.192:3300/0 conn(0x7fef24071710 0x7fef2406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.403+0000 7fef23fff640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0c003970 0x7fef0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef289ee640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef0c004d00 0x7fef0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef237fe640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef23fff640 1 -- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 msgr2=0x7fef0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef23fff640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef241401c0 0x7fef0c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef23fff640 1 -- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef0c004d00 msgr2=0x7fef0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef23fff640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef0c004d00 0x7fef0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef23fff640 1 -- 172.21.15.111:0/2492950066 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef14009400 con 0x7fef0c003970 2023-12-04T05:41:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef23fff640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0c003970 0x7fef0c003d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c90 tx=0x7fef0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffff9053c6f4259 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1402c8f0 con 0x7fef0c003970 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1402ca50 con 0x7fef0c003970 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.404+0000 7fef22ffd640 1 -- 172.21.15.111:0/2492950066 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1402ce00 con 0x7fef0c003970 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef2a451640 1 -- 172.21.15.111:0/2492950066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0c003970 msgr2=0x7fef0c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef2a451640 1 --2- 172.21.15.111:0/2492950066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0c003970 0x7fef0c003d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c90 tx=0x7fef0c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef289ee640 1 -- 172.21.15.111:0/2492950066 reap_dead start 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef2a451640 1 -- 172.21.15.111:0/2492950066 shutdown_connections 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef2a451640 1 -- 172.21.15.111:0/2492950066 >> 172.21.15.111:0/2492950066 conn(0x7fef2406a4c0 msgr2=0x7fef2406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef2a451640 1 -- 172.21.15.111:0/2492950066 shutdown_connections 2023-12-04T05:41:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.405+0000 7fef2a451640 1 -- 172.21.15.111:0/2492950066 wait complete. 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.406+0000 7fef2a451640 1 Processor -- start 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.406+0000 7fef2a451640 1 -- start start 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef2a451640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef0c003970 0x7fef24141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef2a451640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef24134670 0x7fef24157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef2a451640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 0x7fef240b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef2a451640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef240b1c10 con 0x7fef240b1220 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef2a451640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef241f61d0 con 0x7fef24134670 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef2a451640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef241f64b0 con 0x7fef0c003970 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef289ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 0x7fef240b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef289ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 0x7fef240b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57914/0 (socket says 172.21.15.111:57914) 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef289ee640 1 -- 172.21.15.111:0/1737025042 learned_addr learned my addr 172.21.15.111:0/1737025042 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef23fff640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef0c003970 0x7fef24141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef237fe640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef24134670 0x7fef24157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef289ee640 1 -- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef0c003970 msgr2=0x7fef24141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef289ee640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef0c003970 0x7fef24141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.407+0000 7fef289ee640 1 -- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef24134670 msgr2=0x7fef24157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef289ee640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef24134670 0x7fef24157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef289ee640 1 -- 172.21.15.111:0/1737025042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef241f6790 con 0x7fef240b1220 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef289ee640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 0x7fef240b16d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0013d0 tx=0x7fef1c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c007a80 con 0x7fef240b1220 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c00ee20 con 0x7fef240b1220 2023-12-04T05:41:06.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c013660 con 0x7fef240b1220 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef241f6a70 con 0x7fef240b1220 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.408+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef24142b60 con 0x7fef240b1220 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.409+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef241f6310 con 0x7fef240b1220 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.409+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fef1c0137c0 con 0x7fef240b1220 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.409+0000 7fef20ff9640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fef1006b2a0 0x7fef1006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.410+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef1c08a0c0 con 0x7fef240b1220 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.410+0000 7fef23fff640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fef1006b2a0 0x7fef1006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.410+0000 7fef23fff640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fef1006b2a0 0x7fef1006d760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef14036000 tx=0x7fef14030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.414+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef241f6310 con 0x7fef240b1220 2023-12-04T05:41:06.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.580+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 --> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef241f6620 con 0x7fef1006b2a0 2023-12-04T05:41:06.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.587+0000 7fef20ff9640 1 -- 172.21.15.111:0/1737025042 <== mgr.4101 v2:172.21.15.111:6832/3900317453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289982 (secure 0 0 0) 0x7fef241f6620 con 0x7fef1006b2a0 2023-12-04T05:41:06.592 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:06.593 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fef1006b2a0 msgr2=0x7fef1006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fef1006b2a0 0x7fef1006d760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef14036000 tx=0x7fef14030000 comp rx=0 tx=0).stop 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 msgr2=0x7fef240b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 0x7fef240b16d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0013d0 tx=0x7fef1c00e530 comp rx=0 tx=0).stop 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 shutdown_connections 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fef1006b2a0 0x7fef1006d760 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef240b1220 0x7fef240b16d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef24134670 0x7fef24157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 --2- 172.21.15.111:0/1737025042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef0c003970 0x7fef24141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 >> 172.21.15.111:0/1737025042 conn(0x7fef2406a4c0 msgr2=0x7fef2413f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:06.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 shutdown_connections 2023-12-04T05:41:06.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.590+0000 7fef2a451640 1 -- 172.21.15.111:0/1737025042 wait complete. 2023-12-04T05:41:06.602 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2023-12-04T05:41:06.493750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221464,"kb_used_data":5268,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753256,"statfs":{"total":773094113280,"available":772867334144,"internally_reserved":0,"allocated":5394432,"data_stored":2862184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.201087"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.317225+0000","last_change":"2023-12-04T05:41:03.312730+0000","last_active":"2023-12-04T05:41:03.317225+0000","last_peered":"2023-12-04T05:41:03.317225+0000","last_clean":"2023-12-04T05:41:03.317225+0000","last_became_active":"2023-12-04T05:41:03.312343+0000","last_became_peered":"2023-12-04T05:41:03.312343+0000","last_unstale":"2023-12-04T05:41:03.317225+0000","last_undegraded":"2023-12-04T05:41:03.317225+0000","last_fullsized":"2023-12-04T05:41:03.317225+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:27:36.865331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304763+0000","last_change":"2023-12-04T05:41:03.322477+0000","last_active":"2023-12-04T05:41:04.304763+0000","last_peered":"2023-12-04T05:41:04.304763+0000","last_clean":"2023-12-04T05:41:04.304763+0000","last_became_active":"2023-12-04T05:41:03.322199+0000","last_became_peered":"2023-12-04T05:41:03.322199+0000","last_unstale":"2023-12-04T05:41:04.304763+0000","last_undegraded":"2023-12-04T05:41:04.304763+0000","last_fullsized":"2023-12-04T05:41:04.304763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:55:42.030209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305677+0000","last_change":"2023-12-04T05:41:03.318913+0000","last_active":"2023-12-04T05:41:04.305677+0000","last_peered":"2023-12-04T05:41:04.305677+0000","last_clean":"2023-12-04T05:41:04.305677+0000","last_became_active":"2023-12-04T05:41:03.318555+0000","last_became_peered":"2023-12-04T05:41:03.318555+0000","last_unstale":"2023-12-04T05:41:04.305677+0000","last_undegraded":"2023-12-04T05:41:04.305677+0000","last_fullsized":"2023-12-04T05:41:04.305677+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:27:36.865331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.328232+0000","last_change":"2023-12-04T05:41:03.323822+0000","last_active":"2023-12-04T05:41:03.328232+0000","last_peered":"2023-12-04T05:41:03.328232+0000","last_clean":"2023-12-04T05:41:03.328232+0000","last_became_active":"2023-12-04T05:41:03.323217+0000","last_became_peered":"2023-12-04T05:41:03.323217+0000","last_unstale":"2023-12-04T05:41:03.328232+0000","last_undegraded":"2023-12-04T05:41:03.328232+0000","last_fullsized":"2023-12-04T05:41:03.328232+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.364201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304378+0000","last_change":"2023-12-04T05:41:03.322364+0000","last_active":"2023-12-04T05:41:04.304378+0000","last_peered":"2023-12-04T05:41:04.304378+0000","last_clean":"2023-12-04T05:41:04.304378+0000","last_became_active":"2023-12-04T05:41:03.321902+0000","last_became_peered":"2023-12-04T05:41:03.321902+0000","last_unstale":"2023-12-04T05:41:04.304378+0000","last_undegraded":"2023-12-04T05:41:04.304378+0000","last_fullsized":"2023-12-04T05:41:04.304378+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:29:01.951550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.317766+0000","last_change":"2023-12-04T05:41:03.313013+0000","last_active":"2023-12-04T05:41:03.317766+0000","last_peered":"2023-12-04T05:41:03.317766+0000","last_clean":"2023-12-04T05:41:03.317766+0000","last_became_active":"2023-12-04T05:41:03.312748+0000","last_became_peered":"2023-12-04T05:41:03.312748+0000","last_unstale":"2023-12-04T05:41:03.317766+0000","last_undegraded":"2023-12-04T05:41:03.317766+0000","last_fullsized":"2023-12-04T05:41:03.317766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:34:57.639657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.317589+0000","last_change":"2023-12-04T05:41:03.314031+0000","last_active":"2023-12-04T05:41:03.317589+0000","last_peered":"2023-12-04T05:41:03.317589+0000","last_clean":"2023-12-04T05:41:03.317589+0000","last_became_active":"2023-12-04T05:41:03.313768+0000","last_became_peered":"2023-12-04T05:41:03.313768+0000","last_unstale":"2023-12-04T05:41:03.317589+0000","last_undegraded":"2023-12-04T05:41:03.317589+0000","last_fullsized":"2023-12-04T05:41:03.317589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:08:41.081817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.328284+0000","last_change":"2023-12-04T05:41:03.324805+0000","last_active":"2023-12-04T05:41:03.328284+0000","last_peered":"2023-12-04T05:41:03.328284+0000","last_clean":"2023-12-04T05:41:03.328284+0000","last_became_active":"2023-12-04T05:41:03.323773+0000","last_became_peered":"2023-12-04T05:41:03.323773+0000","last_unstale":"2023-12-04T05:41:03.328284+0000","last_undegraded":"2023-12-04T05:41:03.328284+0000","last_fullsized":"2023-12-04T05:41:03.328284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:50:07.158877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.328744+0000","last_change":"2023-12-04T05:41:03.327280+0000","last_active":"2023-12-04T05:41:03.328744+0000","last_peered":"2023-12-04T05:41:03.328744+0000","last_clean":"2023-12-04T05:41:03.328744+0000","last_became_active":"2023-12-04T05:41:03.327020+0000","last_became_peered":"2023-12-04T05:41:03.327020+0000","last_unstale":"2023-12-04T05:41:03.328744+0000","last_undegraded":"2023-12-04T05:41:03.328744+0000","last_fullsized":"2023-12-04T05:41:03.328744+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:19:14.442878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327560+0000","last_change":"2023-12-04T05:41:03.323157+0000","last_active":"2023-12-04T05:41:03.327560+0000","last_peered":"2023-12-04T05:41:03.327560+0000","last_clean":"2023-12-04T05:41:03.327560+0000","last_became_active":"2023-12-04T05:41:03.322784+0000","last_became_peered":"2023-12-04T05:41:03.322784+0000","last_unstale":"2023-12-04T05:41:03.327560+0000","last_undegraded":"2023-12-04T05:41:03.327560+0000","last_fullsized":"2023-12-04T05:41:03.327560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:38.301700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.333575+0000","last_change":"2023-12-04T05:41:03.328512+0000","last_active":"2023-12-04T05:41:03.333575+0000","last_peered":"2023-12-04T05:41:03.333575+0000","last_clean":"2023-12-04T05:41:03.333575+0000","last_became_active":"2023-12-04T05:41:03.328250+0000","last_became_peered":"2023-12-04T05:41:03.328250+0000","last_unstale":"2023-12-04T05:41:03.333575+0000","last_undegraded":"2023-12-04T05:41:03.333575+0000","last_fullsized":"2023-12-04T05:41:03.333575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:48:19.867818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304997+0000","last_change":"2023-12-04T05:41:03.321101+0000","last_active":"2023-12-04T05:41:04.304997+0000","last_peered":"2023-12-04T05:41:04.304997+0000","last_clean":"2023-12-04T05:41:04.304997+0000","last_became_active":"2023-12-04T05:41:03.320693+0000","last_became_peered":"2023-12-04T05:41:03.320693+0000","last_unstale":"2023-12-04T05:41:04.304997+0000","last_undegraded":"2023-12-04T05:41:04.304997+0000","last_fullsized":"2023-12-04T05:41:04.304997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:01:42.810266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.323578+0000","last_change":"2023-12-04T05:41:03.320765+0000","last_active":"2023-12-04T05:41:03.323578+0000","last_peered":"2023-12-04T05:41:03.323578+0000","last_clean":"2023-12-04T05:41:03.323578+0000","last_became_active":"2023-12-04T05:41:03.320527+0000","last_became_peered":"2023-12-04T05:41:03.320527+0000","last_unstale":"2023-12-04T05:41:03.323578+0000","last_undegraded":"2023-12-04T05:41:03.323578+0000","last_fullsized":"2023-12-04T05:41:03.323578+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:54.754439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.333517+0000","last_change":"2023-12-04T05:41:03.328588+0000","last_active":"2023-12-04T05:41:03.333517+0000","last_peered":"2023-12-04T05:41:03.333517+0000","last_clean":"2023-12-04T05:41:03.333517+0000","last_became_active":"2023-12-04T05:41:03.328325+0000","last_became_peered":"2023-12-04T05:41:03.328325+0000","last_unstale":"2023-12-04T05:41:03.333517+0000","last_undegraded":"2023-12-04T05:41:03.333517+0000","last_fullsized":"2023-12-04T05:41:03.333517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:27:47.198390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.322696+0000","last_change":"2023-12-04T05:41:03.315924+0000","last_active":"2023-12-04T05:41:03.322696+0000","last_peered":"2023-12-04T05:41:03.322696+0000","last_clean":"2023-12-04T05:41:03.322696+0000","last_became_active":"2023-12-04T05:41:03.315476+0000","last_became_peered":"2023-12-04T05:41:03.315476+0000","last_unstale":"2023-12-04T05:41:03.322696+0000","last_undegraded":"2023-12-04T05:41:03.322696+0000","last_fullsized":"2023-12-04T05:41:03.322696+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:55:42.030209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.303980+0000","last_change":"2023-12-04T05:41:03.317724+0000","last_active":"2023-12-04T05:41:04.303980+0000","last_peered":"2023-12-04T05:41:04.303980+0000","last_clean":"2023-12-04T05:41:04.303980+0000","last_became_active":"2023-12-04T05:41:03.317295+0000","last_became_peered":"2023-12-04T05:41:03.317295+0000","last_unstale":"2023-12-04T05:41:04.303980+0000","last_undegraded":"2023-12-04T05:41:04.303980+0000","last_fullsized":"2023-12-04T05:41:04.303980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:17:19.490647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305846+0000","last_change":"2023-12-04T05:41:03.312088+0000","last_active":"2023-12-04T05:41:04.305846+0000","last_peered":"2023-12-04T05:41:04.305846+0000","last_clean":"2023-12-04T05:41:04.305846+0000","last_became_active":"2023-12-04T05:41:03.311675+0000","last_became_peered":"2023-12-04T05:41:03.311675+0000","last_unstale":"2023-12-04T05:41:04.305846+0000","last_undegraded":"2023-12-04T05:41:04.305846+0000","last_fullsized":"2023-12-04T05:41:04.305846+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:34:57.639657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305801+0000","last_change":"2023-12-04T05:41:03.317805+0000","last_active":"2023-12-04T05:41:04.305801+0000","last_peered":"2023-12-04T05:41:04.305801+0000","last_clean":"2023-12-04T05:41:04.305801+0000","last_became_active":"2023-12-04T05:41:03.317543+0000","last_became_peered":"2023-12-04T05:41:03.317543+0000","last_unstale":"2023-12-04T05:41:04.305801+0000","last_undegraded":"2023-12-04T05:41:04.305801+0000","last_fullsized":"2023-12-04T05:41:04.305801+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:55:42.030209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304713+0000","last_change":"2023-12-04T05:41:03.319655+0000","last_active":"2023-12-04T05:41:04.304713+0000","last_peered":"2023-12-04T05:41:04.304713+0000","last_clean":"2023-12-04T05:41:04.304713+0000","last_became_active":"2023-12-04T05:41:03.319366+0000","last_became_peered":"2023-12-04T05:41:03.319366+0000","last_unstale":"2023-12-04T05:41:04.304713+0000","last_undegraded":"2023-12-04T05:41:04.304713+0000","last_fullsized":"2023-12-04T05:41:04.304713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:48:19.867818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.323526+0000","last_change":"2023-12-04T05:41:03.319873+0000","last_active":"2023-12-04T05:41:03.323526+0000","last_peered":"2023-12-04T05:41:03.323526+0000","last_clean":"2023-12-04T05:41:03.323526+0000","last_became_active":"2023-12-04T05:41:03.319599+0000","last_became_peered":"2023-12-04T05:41:03.319599+0000","last_unstale":"2023-12-04T05:41:03.323526+0000","last_undegraded":"2023-12-04T05:41:03.323526+0000","last_fullsized":"2023-12-04T05:41:03.323526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:17:12.122731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.324807+0000","last_change":"2023-12-04T05:41:03.317847+0000","last_active":"2023-12-04T05:41:03.324807+0000","last_peered":"2023-12-04T05:41:03.324807+0000","last_clean":"2023-12-04T05:41:03.324807+0000","last_became_active":"2023-12-04T05:41:03.317591+0000","last_became_peered":"2023-12-04T05:41:03.317591+0000","last_unstale":"2023-12-04T05:41:03.324807+0000","last_undegraded":"2023-12-04T05:41:03.324807+0000","last_fullsized":"2023-12-04T05:41:03.324807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:29:01.951550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327587+0000","last_change":"2023-12-04T05:41:03.323567+0000","last_active":"2023-12-04T05:41:03.327587+0000","last_peered":"2023-12-04T05:41:03.327587+0000","last_clean":"2023-12-04T05:41:03.327587+0000","last_became_active":"2023-12-04T05:41:03.323070+0000","last_became_peered":"2023-12-04T05:41:03.323070+0000","last_unstale":"2023-12-04T05:41:03.327587+0000","last_undegraded":"2023-12-04T05:41:03.327587+0000","last_fullsized":"2023-12-04T05:41:03.327587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:31.647851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.303917+0000","last_change":"2023-12-04T05:41:03.319928+0000","last_active":"2023-12-04T05:41:04.303917+0000","last_peered":"2023-12-04T05:41:04.303917+0000","last_clean":"2023-12-04T05:41:04.303917+0000","last_became_active":"2023-12-04T05:41:03.319573+0000","last_became_peered":"2023-12-04T05:41:03.319573+0000","last_unstale":"2023-12-04T05:41:04.303917+0000","last_undegraded":"2023-12-04T05:41:04.303917+0000","last_fullsized":"2023-12-04T05:41:04.303917+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:55:37.385580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304610+0000","last_change":"2023-12-04T05:41:03.313407+0000","last_active":"2023-12-04T05:41:04.304610+0000","last_peered":"2023-12-04T05:41:04.304610+0000","last_clean":"2023-12-04T05:41:04.304610+0000","last_became_active":"2023-12-04T05:41:03.313106+0000","last_became_peered":"2023-12-04T05:41:03.313106+0000","last_unstale":"2023-12-04T05:41:04.304610+0000","last_undegraded":"2023-12-04T05:41:04.304610+0000","last_fullsized":"2023-12-04T05:41:04.304610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:27:47.198390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304117+0000","last_change":"2023-12-04T05:41:03.317146+0000","last_active":"2023-12-04T05:41:04.304117+0000","last_peered":"2023-12-04T05:41:04.304117+0000","last_clean":"2023-12-04T05:41:04.304117+0000","last_became_active":"2023-12-04T05:41:03.316876+0000","last_became_peered":"2023-12-04T05:41:03.316876+0000","last_unstale":"2023-12-04T05:41:04.304117+0000","last_undegraded":"2023-12-04T05:41:04.304117+0000","last_fullsized":"2023-12-04T05:41:04.304117+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:07.364201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.323555+0000","last_change":"2023-12-04T05:41:03.321396+0000","last_active":"2023-12-04T05:41:03.323555+0000","last_peered":"2023-12-04T05:41:03.323555+0000","last_clean":"2023-12-04T05:41:03.323555+0000","last_became_active":"2023-12-04T05:41:03.321147+0000","last_became_peered":"2023-12-04T05:41:03.321147+0000","last_unstale":"2023-12-04T05:41:03.323555+0000","last_undegraded":"2023-12-04T05:41:03.323555+0000","last_fullsized":"2023-12-04T05:41:03.323555+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:13:43.150819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305720+0000","last_change":"2023-12-04T05:41:03.312124+0000","last_active":"2023-12-04T05:41:04.305720+0000","last_peered":"2023-12-04T05:41:04.305720+0000","last_clean":"2023-12-04T05:41:04.305720+0000","last_became_active":"2023-12-04T05:41:03.311856+0000","last_became_peered":"2023-12-04T05:41:03.311856+0000","last_unstale":"2023-12-04T05:41:04.305720+0000","last_undegraded":"2023-12-04T05:41:04.305720+0000","last_fullsized":"2023-12-04T05:41:04.305720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:29:01.951550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305057+0000","last_change":"2023-12-04T05:41:03.320680+0000","last_active":"2023-12-04T05:41:04.305057+0000","last_peered":"2023-12-04T05:41:04.305057+0000","last_clean":"2023-12-04T05:41:04.305057+0000","last_became_active":"2023-12-04T05:41:03.320305+0000","last_became_peered":"2023-12-04T05:41:03.320305+0000","last_unstale":"2023-12-04T05:41:04.305057+0000","last_undegraded":"2023-12-04T05:41:04.305057+0000","last_fullsized":"2023-12-04T05:41:04.305057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:34:51.616104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304894+0000","last_change":"2023-12-04T05:41:03.319727+0000","last_active":"2023-12-04T05:41:04.304894+0000","last_peered":"2023-12-04T05:41:04.304894+0000","last_clean":"2023-12-04T05:41:04.304894+0000","last_became_active":"2023-12-04T05:41:03.319440+0000","last_became_peered":"2023-12-04T05:41:03.319440+0000","last_unstale":"2023-12-04T05:41:04.304894+0000","last_undegraded":"2023-12-04T05:41:04.304894+0000","last_fullsized":"2023-12-04T05:41:04.304894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:38.301700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305541+0000","last_change":"2023-12-04T05:41:03.311640+0000","last_active":"2023-12-04T05:41:04.305541+0000","last_peered":"2023-12-04T05:41:04.305541+0000","last_clean":"2023-12-04T05:41:04.305541+0000","last_became_active":"2023-12-04T05:41:03.311333+0000","last_became_peered":"2023-12-04T05:41:03.311333+0000","last_unstale":"2023-12-04T05:41:04.305541+0000","last_undegraded":"2023-12-04T05:41:04.305541+0000","last_fullsized":"2023-12-04T05:41:04.305541+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:39:09.489235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.322675+0000","last_change":"2023-12-04T05:41:03.315758+0000","last_active":"2023-12-04T05:41:03.322675+0000","last_peered":"2023-12-04T05:41:03.322675+0000","last_clean":"2023-12-04T05:41:03.322675+0000","last_became_active":"2023-12-04T05:41:03.315094+0000","last_became_peered":"2023-12-04T05:41:03.315094+0000","last_unstale":"2023-12-04T05:41:03.322675+0000","last_undegraded":"2023-12-04T05:41:03.322675+0000","last_fullsized":"2023-12-04T05:41:03.322675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:01:42.810266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.321742+0000","last_change":"2023-12-04T05:41:03.317924+0000","last_active":"2023-12-04T05:41:03.321742+0000","last_peered":"2023-12-04T05:41:03.321742+0000","last_clean":"2023-12-04T05:41:03.321742+0000","last_became_active":"2023-12-04T05:41:03.317595+0000","last_became_peered":"2023-12-04T05:41:03.317595+0000","last_unstale":"2023-12-04T05:41:03.321742+0000","last_undegraded":"2023-12-04T05:41:03.321742+0000","last_fullsized":"2023-12-04T05:41:03.321742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:17:33.942364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.332243+0000","last_change":"2023-12-04T05:41:03.326716+0000","last_active":"2023-12-04T05:41:03.332243+0000","last_peered":"2023-12-04T05:41:03.332243+0000","last_clean":"2023-12-04T05:41:03.332243+0000","last_became_active":"2023-12-04T05:41:03.326382+0000","last_became_peered":"2023-12-04T05:41:03.326382+0000","last_unstale":"2023-12-04T05:41:03.332243+0000","last_undegraded":"2023-12-04T05:41:03.332243+0000","last_fullsized":"2023-12-04T05:41:03.332243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:19:13.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.335277+0000","last_change":"2023-12-04T05:41:02.321102+0000","last_active":"2023-12-04T05:41:03.335277+0000","last_peered":"2023-12-04T05:41:03.335277+0000","last_clean":"2023-12-04T05:41:03.335277+0000","last_became_active":"2023-12-04T05:41:02.320619+0000","last_became_peered":"2023-12-04T05:41:02.320619+0000","last_unstale":"2023-12-04T05:41:03.335277+0000","last_undegraded":"2023-12-04T05:41:03.335277+0000","last_fullsized":"2023-12-04T05:41:03.335277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:54:52.245808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304267+0000","last_change":"2023-12-04T05:41:03.314992+0000","last_active":"2023-12-04T05:41:04.304267+0000","last_peered":"2023-12-04T05:41:04.304267+0000","last_clean":"2023-12-04T05:41:04.304267+0000","last_became_active":"2023-12-04T05:41:03.314715+0000","last_became_peered":"2023-12-04T05:41:03.314715+0000","last_unstale":"2023-12-04T05:41:04.304267+0000","last_undegraded":"2023-12-04T05:41:04.304267+0000","last_fullsized":"2023-12-04T05:41:04.304267+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:57:58.501632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305968+0000","last_change":"2023-12-04T05:41:02.313461+0000","last_active":"2023-12-04T05:41:04.305968+0000","last_peered":"2023-12-04T05:41:04.305968+0000","last_clean":"2023-12-04T05:41:04.305968+0000","last_became_active":"2023-12-04T05:41:02.313171+0000","last_became_peered":"2023-12-04T05:41:02.313171+0000","last_unstale":"2023-12-04T05:41:04.305968+0000","last_undegraded":"2023-12-04T05:41:04.305968+0000","last_fullsized":"2023-12-04T05:41:04.305968+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:15:13.207795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305135+0000","last_change":"2023-12-04T05:41:03.315562+0000","last_active":"2023-12-04T05:41:04.305135+0000","last_peered":"2023-12-04T05:41:04.305135+0000","last_clean":"2023-12-04T05:41:04.305135+0000","last_became_active":"2023-12-04T05:41:03.315250+0000","last_became_peered":"2023-12-04T05:41:03.315250+0000","last_unstale":"2023-12-04T05:41:04.305135+0000","last_undegraded":"2023-12-04T05:41:04.305135+0000","last_fullsized":"2023-12-04T05:41:04.305135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:05:52.927874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.334650+0000","last_change":"2023-12-04T05:41:02.316102+0000","last_active":"2023-12-04T05:41:03.334650+0000","last_peered":"2023-12-04T05:41:03.334650+0000","last_clean":"2023-12-04T05:41:03.334650+0000","last_became_active":"2023-12-04T05:41:02.315813+0000","last_became_peered":"2023-12-04T05:41:02.315813+0000","last_unstale":"2023-12-04T05:41:03.334650+0000","last_undegraded":"2023-12-04T05:41:03.334650+0000","last_fullsized":"2023-12-04T05:41:03.334650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:27:46.194310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.325108+0000","last_change":"2023-12-04T05:41:03.321792+0000","last_active":"2023-12-04T05:41:03.325108+0000","last_peered":"2023-12-04T05:41:03.325108+0000","last_clean":"2023-12-04T05:41:03.325108+0000","last_became_active":"2023-12-04T05:41:03.321485+0000","last_became_peered":"2023-12-04T05:41:03.321485+0000","last_unstale":"2023-12-04T05:41:03.325108+0000","last_undegraded":"2023-12-04T05:41:03.325108+0000","last_fullsized":"2023-12-04T05:41:03.325108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:31.647851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.332288+0000","last_change":"2023-12-04T05:41:02.311511+0000","last_active":"2023-12-04T05:41:03.332288+0000","last_peered":"2023-12-04T05:41:03.332288+0000","last_clean":"2023-12-04T05:41:03.332288+0000","last_became_active":"2023-12-04T05:41:02.311203+0000","last_became_peered":"2023-12-04T05:41:02.311203+0000","last_unstale":"2023-12-04T05:41:03.332288+0000","last_undegraded":"2023-12-04T05:41:03.332288+0000","last_fullsized":"2023-12-04T05:41:03.332288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:22:29.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305597+0000","last_change":"2023-12-04T05:41:03.313398+0000","last_active":"2023-12-04T05:41:04.305597+0000","last_peered":"2023-12-04T05:41:04.305597+0000","last_clean":"2023-12-04T05:41:04.305597+0000","last_became_active":"2023-12-04T05:41:03.313129+0000","last_became_peered":"2023-12-04T05:41:03.313129+0000","last_unstale":"2023-12-04T05:41:04.305597+0000","last_undegraded":"2023-12-04T05:41:04.305597+0000","last_fullsized":"2023-12-04T05:41:04.305597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:19:32.199601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.335245+0000","last_change":"2023-12-04T05:41:02.321318+0000","last_active":"2023-12-04T05:41:03.335245+0000","last_peered":"2023-12-04T05:41:03.335245+0000","last_clean":"2023-12-04T05:41:03.335245+0000","last_became_active":"2023-12-04T05:41:02.321046+0000","last_became_peered":"2023-12-04T05:41:02.321046+0000","last_unstale":"2023-12-04T05:41:03.335245+0000","last_undegraded":"2023-12-04T05:41:03.335245+0000","last_fullsized":"2023-12-04T05:41:03.335245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:56:43.077184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327407+0000","last_change":"2023-12-04T05:41:03.322474+0000","last_active":"2023-12-04T05:41:03.327407+0000","last_peered":"2023-12-04T05:41:03.327407+0000","last_clean":"2023-12-04T05:41:03.327407+0000","last_became_active":"2023-12-04T05:41:03.322236+0000","last_became_peered":"2023-12-04T05:41:03.322236+0000","last_unstale":"2023-12-04T05:41:03.327407+0000","last_undegraded":"2023-12-04T05:41:03.327407+0000","last_fullsized":"2023-12-04T05:41:03.327407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:57:58.501632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305650+0000","last_change":"2023-12-04T05:41:02.307917+0000","last_active":"2023-12-04T05:41:04.305650+0000","last_peered":"2023-12-04T05:41:04.305650+0000","last_clean":"2023-12-04T05:41:04.305650+0000","last_became_active":"2023-12-04T05:41:02.307518+0000","last_became_peered":"2023-12-04T05:41:02.307518+0000","last_unstale":"2023-12-04T05:41:04.305650+0000","last_undegraded":"2023-12-04T05:41:04.305650+0000","last_fullsized":"2023-12-04T05:41:04.305650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:22:29.879279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.328263+0000","last_change":"2023-12-04T05:41:03.323276+0000","last_active":"2023-12-04T05:41:03.328263+0000","last_peered":"2023-12-04T05:41:03.328263+0000","last_clean":"2023-12-04T05:41:03.328263+0000","last_became_active":"2023-12-04T05:41:03.322952+0000","last_became_peered":"2023-12-04T05:41:03.322952+0000","last_unstale":"2023-12-04T05:41:03.328263+0000","last_undegraded":"2023-12-04T05:41:03.328263+0000","last_fullsized":"2023-12-04T05:41:03.328263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:53:41.294717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304236+0000","last_change":"2023-12-04T05:41:02.312379+0000","last_active":"2023-12-04T05:41:04.304236+0000","last_peered":"2023-12-04T05:41:04.304236+0000","last_clean":"2023-12-04T05:41:04.304236+0000","last_became_active":"2023-12-04T05:41:02.312057+0000","last_became_peered":"2023-12-04T05:41:02.312057+0000","last_unstale":"2023-12-04T05:41:04.304236+0000","last_undegraded":"2023-12-04T05:41:04.304236+0000","last_fullsized":"2023-12-04T05:41:04.304236+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:51:04.210990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304370+0000","last_change":"2023-12-04T05:41:03.320241+0000","last_active":"2023-12-04T05:41:04.304370+0000","last_peered":"2023-12-04T05:41:04.304370+0000","last_clean":"2023-12-04T05:41:04.304370+0000","last_became_active":"2023-12-04T05:41:03.319968+0000","last_became_peered":"2023-12-04T05:41:03.319968+0000","last_unstale":"2023-12-04T05:41:04.304370+0000","last_undegraded":"2023-12-04T05:41:04.304370+0000","last_fullsized":"2023-12-04T05:41:04.304370+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:31.647851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.310743+0000","last_change":"2023-12-04T05:41:02.307603+0000","last_active":"2023-12-04T05:41:03.310743+0000","last_peered":"2023-12-04T05:41:03.310743+0000","last_clean":"2023-12-04T05:41:03.310743+0000","last_became_active":"2023-12-04T05:41:02.307347+0000","last_became_peered":"2023-12-04T05:41:02.307347+0000","last_unstale":"2023-12-04T05:41:03.310743+0000","last_undegraded":"2023-12-04T05:41:03.310743+0000","last_fullsized":"2023-12-04T05:41:03.310743+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:32:50.399359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305869+0000","last_change":"2023-12-04T05:41:03.315599+0000","last_active":"2023-12-04T05:41:04.305869+0000","last_peered":"2023-12-04T05:41:04.305869+0000","last_clean":"2023-12-04T05:41:04.305869+0000","last_became_active":"2023-12-04T05:41:03.315346+0000","last_became_peered":"2023-12-04T05:41:03.315346+0000","last_unstale":"2023-12-04T05:41:04.305869+0000","last_undegraded":"2023-12-04T05:41:04.305869+0000","last_fullsized":"2023-12-04T05:41:04.305869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:01:42.810266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305957+0000","last_change":"2023-12-04T05:41:02.314934+0000","last_active":"2023-12-04T05:41:04.305957+0000","last_peered":"2023-12-04T05:41:04.305957+0000","last_clean":"2023-12-04T05:41:04.305957+0000","last_became_active":"2023-12-04T05:41:02.314600+0000","last_became_peered":"2023-12-04T05:41:02.314600+0000","last_unstale":"2023-12-04T05:41:04.305957+0000","last_undegraded":"2023-12-04T05:41:04.305957+0000","last_fullsized":"2023-12-04T05:41:04.305957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:27:46.194310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.317205+0000","last_change":"2023-12-04T05:41:03.312598+0000","last_active":"2023-12-04T05:41:03.317205+0000","last_peered":"2023-12-04T05:41:03.317205+0000","last_clean":"2023-12-04T05:41:03.317205+0000","last_became_active":"2023-12-04T05:41:03.311995+0000","last_became_peered":"2023-12-04T05:41:03.311995+0000","last_unstale":"2023-12-04T05:41:03.317205+0000","last_undegraded":"2023-12-04T05:41:03.317205+0000","last_fullsized":"2023-12-04T05:41:03.317205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:55:42.030209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.312384+0000","last_change":"2023-12-04T05:41:02.312074+0000","last_active":"2023-12-04T05:41:03.312384+0000","last_peered":"2023-12-04T05:41:03.312384+0000","last_clean":"2023-12-04T05:41:03.312384+0000","last_became_active":"2023-12-04T05:41:02.311786+0000","last_became_peered":"2023-12-04T05:41:02.311786+0000","last_unstale":"2023-12-04T05:41:03.312384+0000","last_undegraded":"2023-12-04T05:41:03.312384+0000","last_fullsized":"2023-12-04T05:41:03.312384+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:08:40.077737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.332616+0000","last_change":"2023-12-04T05:41:03.324627+0000","last_active":"2023-12-04T05:41:03.332616+0000","last_peered":"2023-12-04T05:41:03.332616+0000","last_clean":"2023-12-04T05:41:03.332616+0000","last_became_active":"2023-12-04T05:41:03.324341+0000","last_became_peered":"2023-12-04T05:41:03.324341+0000","last_unstale":"2023-12-04T05:41:03.332616+0000","last_undegraded":"2023-12-04T05:41:03.332616+0000","last_fullsized":"2023-12-04T05:41:03.332616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:19:08.606775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.332310+0000","last_change":"2023-12-04T05:41:02.315388+0000","last_active":"2023-12-04T05:41:03.332310+0000","last_peered":"2023-12-04T05:41:03.332310+0000","last_clean":"2023-12-04T05:41:03.332310+0000","last_became_active":"2023-12-04T05:41:02.315125+0000","last_became_peered":"2023-12-04T05:41:02.315125+0000","last_unstale":"2023-12-04T05:41:03.332310+0000","last_undegraded":"2023-12-04T05:41:03.332310+0000","last_fullsized":"2023-12-04T05:41:03.332310+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:39:08.485155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.318002+0000","last_change":"2023-12-04T05:41:03.316912+0000","last_active":"2023-12-04T05:41:03.318002+0000","last_peered":"2023-12-04T05:41:03.318002+0000","last_clean":"2023-12-04T05:41:03.318002+0000","last_became_active":"2023-12-04T05:41:03.316638+0000","last_became_peered":"2023-12-04T05:41:03.316638+0000","last_unstale":"2023-12-04T05:41:03.318002+0000","last_undegraded":"2023-12-04T05:41:03.318002+0000","last_fullsized":"2023-12-04T05:41:03.318002+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:38.301700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.340085+0000","last_change":"2023-12-04T05:41:02.314633+0000","last_active":"2023-12-04T05:41:03.340085+0000","last_peered":"2023-12-04T05:41:03.340085+0000","last_clean":"2023-12-04T05:41:03.340085+0000","last_became_active":"2023-12-04T05:41:02.314363+0000","last_became_peered":"2023-12-04T05:41:02.314363+0000","last_unstale":"2023-12-04T05:41:03.340085+0000","last_undegraded":"2023-12-04T05:41:03.340085+0000","last_fullsized":"2023-12-04T05:41:03.340085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:20:10.873919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.303981+0000","last_change":"2023-12-04T05:41:03.320458+0000","last_active":"2023-12-04T05:41:04.303981+0000","last_peered":"2023-12-04T05:41:04.303981+0000","last_clean":"2023-12-04T05:41:04.303981+0000","last_became_active":"2023-12-04T05:41:03.320150+0000","last_became_peered":"2023-12-04T05:41:03.320150+0000","last_unstale":"2023-12-04T05:41:04.303981+0000","last_undegraded":"2023-12-04T05:41:04.303981+0000","last_fullsized":"2023-12-04T05:41:04.303981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:19:18.910583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.334006+0000","last_change":"2023-12-04T05:41:02.320816+0000","last_active":"2023-12-04T05:41:03.334006+0000","last_peered":"2023-12-04T05:41:03.334006+0000","last_clean":"2023-12-04T05:41:03.334006+0000","last_became_active":"2023-12-04T05:41:02.320521+0000","last_became_peered":"2023-12-04T05:41:02.320521+0000","last_unstale":"2023-12-04T05:41:03.334006+0000","last_undegraded":"2023-12-04T05:41:03.334006+0000","last_fullsized":"2023-12-04T05:41:03.334006+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:33:42.696152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327390+0000","last_change":"2023-12-04T05:41:03.320013+0000","last_active":"2023-12-04T05:41:03.327390+0000","last_peered":"2023-12-04T05:41:03.327390+0000","last_clean":"2023-12-04T05:41:03.327390+0000","last_became_active":"2023-12-04T05:41:03.319702+0000","last_became_peered":"2023-12-04T05:41:03.319702+0000","last_unstale":"2023-12-04T05:41:03.327390+0000","last_undegraded":"2023-12-04T05:41:03.327390+0000","last_fullsized":"2023-12-04T05:41:03.327390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:19:13.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.302716+0000","last_change":"2023-12-04T05:41:02.313530+0000","last_active":"2023-12-04T05:41:03.302716+0000","last_peered":"2023-12-04T05:41:03.302716+0000","last_clean":"2023-12-04T05:41:03.302716+0000","last_became_active":"2023-12-04T05:41:02.313251+0000","last_became_peered":"2023-12-04T05:41:02.313251+0000","last_unstale":"2023-12-04T05:41:03.302716+0000","last_undegraded":"2023-12-04T05:41:03.302716+0000","last_fullsized":"2023-12-04T05:41:03.302716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:48:18.863738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327922+0000","last_change":"2023-12-04T05:41:03.323069+0000","last_active":"2023-12-04T05:41:03.327922+0000","last_peered":"2023-12-04T05:41:03.327922+0000","last_clean":"2023-12-04T05:41:03.327922+0000","last_became_active":"2023-12-04T05:41:03.322805+0000","last_became_peered":"2023-12-04T05:41:03.322805+0000","last_unstale":"2023-12-04T05:41:03.327922+0000","last_undegraded":"2023-12-04T05:41:03.327922+0000","last_fullsized":"2023-12-04T05:41:03.327922+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:27:36.865331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.305269+0000","last_change":"2023-12-04T05:41:02.311749+0000","last_active":"2023-12-04T05:41:03.305269+0000","last_peered":"2023-12-04T05:41:03.305269+0000","last_clean":"2023-12-04T05:41:03.305269+0000","last_became_active":"2023-12-04T05:41:02.311483+0000","last_became_peered":"2023-12-04T05:41:02.311483+0000","last_unstale":"2023-12-04T05:41:03.305269+0000","last_undegraded":"2023-12-04T05:41:03.305269+0000","last_fullsized":"2023-12-04T05:41:03.305269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:10:02.735901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.317604+0000","last_change":"2023-12-04T05:41:03.316564+0000","last_active":"2023-12-04T05:41:03.317604+0000","last_peered":"2023-12-04T05:41:03.317604+0000","last_clean":"2023-12-04T05:41:03.317604+0000","last_became_active":"2023-12-04T05:41:03.316234+0000","last_became_peered":"2023-12-04T05:41:03.316234+0000","last_unstale":"2023-12-04T05:41:03.317604+0000","last_undegraded":"2023-12-04T05:41:03.317604+0000","last_fullsized":"2023-12-04T05:41:03.317604+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:51:05.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.338804+0000","last_change":"2023-12-04T05:41:02.311919+0000","last_active":"2023-12-04T05:41:03.338804+0000","last_peered":"2023-12-04T05:41:03.338804+0000","last_clean":"2023-12-04T05:41:03.338804+0000","last_became_active":"2023-12-04T05:41:02.311630+0000","last_became_peered":"2023-12-04T05:41:02.311630+0000","last_unstale":"2023-12-04T05:41:03.338804+0000","last_undegraded":"2023-12-04T05:41:03.338804+0000","last_fullsized":"2023-12-04T05:41:03.338804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:17:18.486567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.325033+0000","last_change":"2023-12-04T05:41:03.323109+0000","last_active":"2023-12-04T05:41:03.325033+0000","last_peered":"2023-12-04T05:41:03.325033+0000","last_clean":"2023-12-04T05:41:03.325033+0000","last_became_active":"2023-12-04T05:41:03.322845+0000","last_became_peered":"2023-12-04T05:41:03.322845+0000","last_unstale":"2023-12-04T05:41:03.325033+0000","last_undegraded":"2023-12-04T05:41:03.325033+0000","last_fullsized":"2023-12-04T05:41:03.325033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:48:23.066449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.333862+0000","last_change":"2023-12-04T05:41:02.323737+0000","last_active":"2023-12-04T05:41:03.333862+0000","last_peered":"2023-12-04T05:41:03.333862+0000","last_clean":"2023-12-04T05:41:03.333862+0000","last_became_active":"2023-12-04T05:41:02.323449+0000","last_became_peered":"2023-12-04T05:41:02.323449+0000","last_unstale":"2023-12-04T05:41:03.333862+0000","last_undegraded":"2023-12-04T05:41:03.333862+0000","last_fullsized":"2023-12-04T05:41:03.333862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:11:56.576261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.332941+0000","last_change":"2023-12-04T05:41:03.327819+0000","last_active":"2023-12-04T05:41:03.332941+0000","last_peered":"2023-12-04T05:41:03.332941+0000","last_clean":"2023-12-04T05:41:03.332941+0000","last_became_active":"2023-12-04T05:41:03.327575+0000","last_became_peered":"2023-12-04T05:41:03.327575+0000","last_unstale":"2023-12-04T05:41:03.332941+0000","last_undegraded":"2023-12-04T05:41:03.332941+0000","last_fullsized":"2023-12-04T05:41:03.332941+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:33:03.958176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305878+0000","last_change":"2023-12-04T05:41:02.320915+0000","last_active":"2023-12-04T05:41:04.305878+0000","last_peered":"2023-12-04T05:41:04.305878+0000","last_clean":"2023-12-04T05:41:04.305878+0000","last_became_active":"2023-12-04T05:41:02.320610+0000","last_became_peered":"2023-12-04T05:41:02.320610+0000","last_unstale":"2023-12-04T05:41:04.305878+0000","last_undegraded":"2023-12-04T05:41:04.305878+0000","last_fullsized":"2023-12-04T05:41:04.305878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:08:40.077737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304108+0000","last_change":"2023-12-04T05:41:03.319004+0000","last_active":"2023-12-04T05:41:04.304108+0000","last_peered":"2023-12-04T05:41:04.304108+0000","last_clean":"2023-12-04T05:41:04.304108+0000","last_became_active":"2023-12-04T05:41:03.318791+0000","last_became_peered":"2023-12-04T05:41:03.318791+0000","last_unstale":"2023-12-04T05:41:04.304108+0000","last_undegraded":"2023-12-04T05:41:04.304108+0000","last_fullsized":"2023-12-04T05:41:04.304108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:27:18.095547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304479+0000","last_change":"2023-12-04T05:41:02.320768+0000","last_active":"2023-12-04T05:41:04.304479+0000","last_peered":"2023-12-04T05:41:04.304479+0000","last_clean":"2023-12-04T05:41:04.304479+0000","last_became_active":"2023-12-04T05:41:02.320497+0000","last_became_peered":"2023-12-04T05:41:02.320497+0000","last_unstale":"2023-12-04T05:41:04.304479+0000","last_undegraded":"2023-12-04T05:41:04.304479+0000","last_fullsized":"2023-12-04T05:41:04.304479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:39:08.485155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304750+0000","last_change":"2023-12-04T05:41:03.324670+0000","last_active":"2023-12-04T05:41:04.304750+0000","last_peered":"2023-12-04T05:41:04.304750+0000","last_clean":"2023-12-04T05:41:04.304750+0000","last_became_active":"2023-12-04T05:41:03.324412+0000","last_became_peered":"2023-12-04T05:41:03.324412+0000","last_unstale":"2023-12-04T05:41:04.304750+0000","last_undegraded":"2023-12-04T05:41:04.304750+0000","last_fullsized":"2023-12-04T05:41:04.304750+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:22:30.883359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.305311+0000","last_change":"2023-12-04T05:41:02.321137+0000","last_active":"2023-12-04T05:41:03.305311+0000","last_peered":"2023-12-04T05:41:03.305311+0000","last_clean":"2023-12-04T05:41:03.305311+0000","last_became_active":"2023-12-04T05:41:02.320571+0000","last_became_peered":"2023-12-04T05:41:02.320571+0000","last_unstale":"2023-12-04T05:41:03.305311+0000","last_undegraded":"2023-12-04T05:41:03.305311+0000","last_fullsized":"2023-12-04T05:41:03.305311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T10:18:30.926389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.328098+0000","last_change":"2023-12-04T05:41:03.323417+0000","last_active":"2023-12-04T05:41:03.328098+0000","last_peered":"2023-12-04T05:41:03.328098+0000","last_clean":"2023-12-04T05:41:03.328098+0000","last_became_active":"2023-12-04T05:41:03.323165+0000","last_became_peered":"2023-12-04T05:41:03.323165+0000","last_unstale":"2023-12-04T05:41:03.328098+0000","last_undegraded":"2023-12-04T05:41:03.328098+0000","last_fullsized":"2023-12-04T05:41:03.328098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:11:02.472479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.335329+0000","last_change":"2023-12-04T05:41:02.321529+0000","last_active":"2023-12-04T05:41:03.335329+0000","last_peered":"2023-12-04T05:41:03.335329+0000","last_clean":"2023-12-04T05:41:03.335329+0000","last_became_active":"2023-12-04T05:41:02.321285+0000","last_became_peered":"2023-12-04T05:41:02.321285+0000","last_unstale":"2023-12-04T05:41:03.335329+0000","last_undegraded":"2023-12-04T05:41:03.335329+0000","last_fullsized":"2023-12-04T05:41:03.335329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T13:34:59.695664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.324323+0000","last_change":"2023-12-04T05:41:03.315842+0000","last_active":"2023-12-04T05:41:03.324323+0000","last_peered":"2023-12-04T05:41:03.324323+0000","last_clean":"2023-12-04T05:41:03.324323+0000","last_became_active":"2023-12-04T05:41:03.315555+0000","last_became_peered":"2023-12-04T05:41:03.315555+0000","last_unstale":"2023-12-04T05:41:03.324323+0000","last_undegraded":"2023-12-04T05:41:03.324323+0000","last_fullsized":"2023-12-04T05:41:03.324323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:27:18.095547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304695+0000","last_change":"2023-12-04T05:41:02.313813+0000","last_active":"2023-12-04T05:41:04.304695+0000","last_peered":"2023-12-04T05:41:04.304695+0000","last_clean":"2023-12-04T05:41:04.304695+0000","last_became_active":"2023-12-04T05:41:02.313203+0000","last_became_peered":"2023-12-04T05:41:02.313203+0000","last_unstale":"2023-12-04T05:41:04.304695+0000","last_undegraded":"2023-12-04T05:41:04.304695+0000","last_fullsized":"2023-12-04T05:41:04.304695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:15:13.207795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.323539+0000","last_change":"2023-12-04T05:41:03.320159+0000","last_active":"2023-12-04T05:41:03.323539+0000","last_peered":"2023-12-04T05:41:03.323539+0000","last_clean":"2023-12-04T05:41:03.323539+0000","last_became_active":"2023-12-04T05:41:03.319885+0000","last_became_peered":"2023-12-04T05:41:03.319885+0000","last_unstale":"2023-12-04T05:41:03.323539+0000","last_undegraded":"2023-12-04T05:41:03.323539+0000","last_fullsized":"2023-12-04T05:41:03.323539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T06:45:20.279829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.308672+0000","last_change":"2023-12-04T05:41:02.311490+0000","last_active":"2023-12-04T05:41:03.308672+0000","last_peered":"2023-12-04T05:41:03.308672+0000","last_clean":"2023-12-04T05:41:03.308672+0000","last_became_active":"2023-12-04T05:41:02.311216+0000","last_became_peered":"2023-12-04T05:41:02.311216+0000","last_unstale":"2023-12-04T05:41:03.308672+0000","last_undegraded":"2023-12-04T05:41:03.308672+0000","last_fullsized":"2023-12-04T05:41:03.308672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:55:36.381500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327298+0000","last_change":"2023-12-04T05:41:03.320366+0000","last_active":"2023-12-04T05:41:03.327298+0000","last_peered":"2023-12-04T05:41:03.327298+0000","last_clean":"2023-12-04T05:41:03.327298+0000","last_became_active":"2023-12-04T05:41:03.320101+0000","last_became_peered":"2023-12-04T05:41:03.320101+0000","last_unstale":"2023-12-04T05:41:03.327298+0000","last_undegraded":"2023-12-04T05:41:03.327298+0000","last_fullsized":"2023-12-04T05:41:03.327298+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:16:14.415999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.302511+0000","last_change":"2023-12-04T05:41:02.316487+0000","last_active":"2023-12-04T05:41:03.302511+0000","last_peered":"2023-12-04T05:41:03.302511+0000","last_clean":"2023-12-04T05:41:03.302511+0000","last_became_active":"2023-12-04T05:41:02.316087+0000","last_became_peered":"2023-12-04T05:41:02.316087+0000","last_unstale":"2023-12-04T05:41:03.302511+0000","last_undegraded":"2023-12-04T05:41:03.302511+0000","last_fullsized":"2023-12-04T05:41:03.302511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:27:46.194310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304409+0000","last_change":"2023-12-04T05:41:03.320376+0000","last_active":"2023-12-04T05:41:04.304409+0000","last_peered":"2023-12-04T05:41:04.304409+0000","last_clean":"2023-12-04T05:41:04.304409+0000","last_became_active":"2023-12-04T05:41:03.320064+0000","last_became_peered":"2023-12-04T05:41:03.320064+0000","last_unstale":"2023-12-04T05:41:04.304409+0000","last_undegraded":"2023-12-04T05:41:04.304409+0000","last_fullsized":"2023-12-04T05:41:04.304409+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:19:13.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.305078+0000","last_change":"2023-12-04T05:41:02.314575+0000","last_active":"2023-12-04T05:41:03.305078+0000","last_peered":"2023-12-04T05:41:03.305078+0000","last_clean":"2023-12-04T05:41:03.305078+0000","last_became_active":"2023-12-04T05:41:02.314273+0000","last_became_peered":"2023-12-04T05:41:02.314273+0000","last_unstale":"2023-12-04T05:41:03.305078+0000","last_undegraded":"2023-12-04T05:41:03.305078+0000","last_fullsized":"2023-12-04T05:41:03.305078+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:05:51.923794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.325848+0000","last_change":"2023-12-04T05:41:03.318603+0000","last_active":"2023-12-04T05:41:03.325848+0000","last_peered":"2023-12-04T05:41:03.325848+0000","last_clean":"2023-12-04T05:41:03.325848+0000","last_became_active":"2023-12-04T05:41:03.318329+0000","last_became_peered":"2023-12-04T05:41:03.318329+0000","last_unstale":"2023-12-04T05:41:03.325848+0000","last_undegraded":"2023-12-04T05:41:03.325848+0000","last_fullsized":"2023-12-04T05:41:03.325848+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:19:18.910583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.303984+0000","last_change":"2023-12-04T05:41:02.315671+0000","last_active":"2023-12-04T05:41:03.303984+0000","last_peered":"2023-12-04T05:41:03.303984+0000","last_clean":"2023-12-04T05:41:03.303984+0000","last_became_active":"2023-12-04T05:41:02.315366+0000","last_became_peered":"2023-12-04T05:41:02.315366+0000","last_unstale":"2023-12-04T05:41:03.303984+0000","last_undegraded":"2023-12-04T05:41:03.303984+0000","last_fullsized":"2023-12-04T05:41:03.303984+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:19:31.195521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.325978+0000","last_change":"2023-12-04T05:41:03.321660+0000","last_active":"2023-12-04T05:41:03.325978+0000","last_peered":"2023-12-04T05:41:03.325978+0000","last_clean":"2023-12-04T05:41:03.325978+0000","last_became_active":"2023-12-04T05:41:03.321371+0000","last_became_peered":"2023-12-04T05:41:03.321371+0000","last_unstale":"2023-12-04T05:41:03.325978+0000","last_undegraded":"2023-12-04T05:41:03.325978+0000","last_fullsized":"2023-12-04T05:41:03.325978+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:15:14.211875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.310324+0000","last_change":"2023-12-04T05:41:02.315386+0000","last_active":"2023-12-04T05:41:03.310324+0000","last_peered":"2023-12-04T05:41:03.310324+0000","last_clean":"2023-12-04T05:41:03.310324+0000","last_became_active":"2023-12-04T05:41:02.315115+0000","last_became_peered":"2023-12-04T05:41:02.315115+0000","last_unstale":"2023-12-04T05:41:03.310324+0000","last_undegraded":"2023-12-04T05:41:03.310324+0000","last_fullsized":"2023-12-04T05:41:03.310324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T07:38:06.360121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.324415+0000","last_change":"2023-12-04T05:41:03.316953+0000","last_active":"2023-12-04T05:41:03.324415+0000","last_peered":"2023-12-04T05:41:03.324415+0000","last_clean":"2023-12-04T05:41:03.324415+0000","last_became_active":"2023-12-04T05:41:03.316690+0000","last_became_peered":"2023-12-04T05:41:03.316690+0000","last_unstale":"2023-12-04T05:41:03.324415+0000","last_undegraded":"2023-12-04T05:41:03.324415+0000","last_fullsized":"2023-12-04T05:41:03.324415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T15:34:51.616104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.304088+0000","last_change":"2023-12-04T05:41:02.315376+0000","last_active":"2023-12-04T05:41:03.304088+0000","last_peered":"2023-12-04T05:41:03.304088+0000","last_clean":"2023-12-04T05:41:03.304088+0000","last_became_active":"2023-12-04T05:41:02.315076+0000","last_became_peered":"2023-12-04T05:41:02.315076+0000","last_unstale":"2023-12-04T05:41:03.304088+0000","last_undegraded":"2023-12-04T05:41:03.304088+0000","last_fullsized":"2023-12-04T05:41:03.304088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:29:00.947470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.332656+0000","last_change":"2023-12-04T05:41:03.325566+0000","last_active":"2023-12-04T05:41:03.332656+0000","last_peered":"2023-12-04T05:41:03.332656+0000","last_clean":"2023-12-04T05:41:03.332656+0000","last_became_active":"2023-12-04T05:41:03.324647+0000","last_became_peered":"2023-12-04T05:41:03.324647+0000","last_unstale":"2023-12-04T05:41:03.332656+0000","last_undegraded":"2023-12-04T05:41:03.332656+0000","last_fullsized":"2023-12-04T05:41:03.332656+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T14:03:04.138572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305735+0000","last_change":"2023-12-04T05:41:02.321342+0000","last_active":"2023-12-04T05:41:04.305735+0000","last_peered":"2023-12-04T05:41:04.305735+0000","last_clean":"2023-12-04T05:41:04.305735+0000","last_became_active":"2023-12-04T05:41:02.321048+0000","last_became_peered":"2023-12-04T05:41:02.321048+0000","last_unstale":"2023-12-04T05:41:04.305735+0000","last_undegraded":"2023-12-04T05:41:04.305735+0000","last_fullsized":"2023-12-04T05:41:04.305735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T11:45:37.297620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304951+0000","last_change":"2023-12-04T05:41:03.314733+0000","last_active":"2023-12-04T05:41:04.304951+0000","last_peered":"2023-12-04T05:41:04.304951+0000","last_clean":"2023-12-04T05:41:04.304951+0000","last_became_active":"2023-12-04T05:41:03.314473+0000","last_became_peered":"2023-12-04T05:41:03.314473+0000","last_unstale":"2023-12-04T05:41:04.304951+0000","last_undegraded":"2023-12-04T05:41:04.304951+0000","last_fullsized":"2023-12-04T05:41:04.304951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T16:34:57.639657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.310397+0000","last_change":"2023-12-04T05:41:02.316140+0000","last_active":"2023-12-04T05:41:03.310397+0000","last_peered":"2023-12-04T05:41:03.310397+0000","last_clean":"2023-12-04T05:41:03.310397+0000","last_became_active":"2023-12-04T05:41:02.315858+0000","last_became_peered":"2023-12-04T05:41:02.315858+0000","last_unstale":"2023-12-04T05:41:03.310397+0000","last_undegraded":"2023-12-04T05:41:03.310397+0000","last_fullsized":"2023-12-04T05:41:03.310397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:57:57.497552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.305517+0000","last_change":"2023-12-04T05:41:03.306956+0000","last_active":"2023-12-04T05:41:04.305517+0000","last_peered":"2023-12-04T05:41:04.305517+0000","last_clean":"2023-12-04T05:41:04.305517+0000","last_became_active":"2023-12-04T05:41:03.306553+0000","last_became_peered":"2023-12-04T05:41:03.306553+0000","last_unstale":"2023-12-04T05:41:04.305517+0000","last_undegraded":"2023-12-04T05:41:04.305517+0000","last_fullsized":"2023-12-04T05:41:04.305517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T08:51:05.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.308970+0000","last_change":"2023-12-04T05:41:02.308126+0000","last_active":"2023-12-04T05:41:03.308970+0000","last_peered":"2023-12-04T05:41:03.308970+0000","last_clean":"2023-12-04T05:41:03.308970+0000","last_became_active":"2023-12-04T05:41:02.307850+0000","last_became_peered":"2023-12-04T05:41:02.307850+0000","last_unstale":"2023-12-04T05:41:03.308970+0000","last_undegraded":"2023-12-04T05:41:03.308970+0000","last_fullsized":"2023-12-04T05:41:03.308970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T17:27:17.091467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-12-04T05:41:03.327259+0000","last_change":"2023-12-04T05:41:03.319399+0000","last_active":"2023-12-04T05:41:03.327259+0000","last_peered":"2023-12-04T05:41:03.327259+0000","last_clean":"2023-12-04T05:41:03.327259+0000","last_became_active":"2023-12-04T05:41:03.318952+0000","last_became_peered":"2023-12-04T05:41:03.318952+0000","last_unstale":"2023-12-04T05:41:03.327259+0000","last_undegraded":"2023-12-04T05:41:03.327259+0000","last_fullsized":"2023-12-04T05:41:03.327259+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:02.292103+0000","last_clean_scrub_stamp":"2023-12-04T05:41:02.292103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T09:22:30.883359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-12-04T05:41:04.304746+0000","last_change":"2023-12-04T05:41:02.321219+0000","last_active":"2023-12-04T05:41:04.304746+0000","last_peered":"2023-12-04T05:41:04.304746+0000","last_clean":"2023-12-04T05:41:04.304746+0000","last_became_active":"2023-12-04T05:41:02.320893+0000","last_became_peered":"2023-12-04T05:41:02.320893+0000","last_unstale":"2023-12-04T05:41:04.304746+0000","last_undegraded":"2023-12-04T05:41:04.304746+0000","last_fullsized":"2023-12-04T05:41:04.304746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:41:01.288023+0000","last_clean_scrub_stamp":"2023-12-04T05:41:01.288023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:19:31.195521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":185,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-12-04T05:41:02.306694+0000","last_change":"2023-12-04T05:39:05.756808+0000","last_active":"2023-12-04T05:41:02.306694+0000","last_peered":"2023-12-04T05:41:02.306694+0000","last_clean":"2023-12-04T05:41:02.306694+0000","last_became_active":"2023-12-04T05:39:05.755841+0000","last_became_peered":"2023-12-04T05:39:05.755841+0000","last_unstale":"2023-12-04T05:41:02.306694+0000","last_undegraded":"2023-12-04T05:41:02.306694+0000","last_fullsized":"2023-12-04T05:41:02.306694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-04T05:39:04.556625+0000","last_clean_scrub_stamp":"2023-12-04T05:39:04.556625+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-05T12:30:26.915697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":799249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.875,"5min":3.875,"15min":3.875},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.613,"5min":3.613,"15min":3.613},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.04,"5min":4.04,"15min":4.04},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":0.47299999999999998}]},{"osd":3,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.2389999999999999,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55000000000000004}]},{"osd":6,"last update":"Mon Dec 4 05:40:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":215763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71399999999999997}]},{"osd":1,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.625},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.501},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Mon Dec 4 05:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.56499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":208861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.45800000000000002}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.41299999999999998}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47799999999999998}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.46800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":528384,"data_stored":210403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47199999999999998}]},{"osd":2,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.41599999999999998}]},{"osd":5,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.626},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.49399999999999999}]},{"osd":7,"last update":"Mon Dec 4 05:40:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.60099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":528384,"data_stored":209429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.36299999999999999}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.312},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40999999999999998}]},{"osd":2,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64200000000000002}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":210425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.372,"5min":0.372,"15min":0.372},"last":0.51500000000000001}]},{"osd":1,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.441}]},{"osd":4,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.622},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Dec 4 05:40:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.747},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":208861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.36699999999999999}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.34200000000000003}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.623}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":799193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.308},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.249}]},{"osd":2,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.29299999999999998}]},{"osd":3,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.501},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.65600000000000003}]},{"osd":5,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Dec 4 05:40:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.628},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.47599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-04T05:41:06.607 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-04T05:41:06.607 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-04T05:41:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.748+0000 7f51c5cad640 1 Processor -- start 2023-12-04T05:41:06.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.748+0000 7f51c5cad640 1 -- start start 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51c5cad640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c00b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51c5cad640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f51c0072250 0x7f51c0070510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51c5cad640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c015c420 con 0x7f51c015c770 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51c5cad640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c00b4c20 con 0x7f51c0072250 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51c5cad640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00ced20 con 0x7f51c00b4300 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f51c0072250 0x7f51c0070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f51c0072250 0x7f51c0070510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57916/0 (socket says 172.21.15.111:57916) 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 -- 172.21.15.111:0/1279103400 learned_addr learned my addr 172.21.15.111:0/1279103400 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bf7fe640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c00b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 -- 172.21.15.111:0/1279103400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 msgr2=0x7f51c00b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c00b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 -- 172.21.15.111:0/1279103400 >> v1:172.21.15.192:6789/0 conn(0x7f51c015c770 legacy=0x7f51c0071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bffff640 1 -- 172.21.15.111:0/1279103400 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c015b810 con 0x7f51c0072250 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.749+0000 7f51bf7fe640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c00b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51bffff640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c0072250 0x7f51c0070510 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009e50 tx=0x7f51b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a4113e36415603 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51be7fc640 1 -- 172.21.15.111:0/1279103400 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b802ca70 con 0x7f51c0072250 2023-12-04T05:41:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51be7fc640 1 -- 172.21.15.111:0/1279103400 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b802cbd0 con 0x7f51c0072250 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51be7fc640 1 -- 172.21.15.111:0/1279103400 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80343f0 con 0x7f51c0072250 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51c5cad640 1 -- 172.21.15.111:0/1279103400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c0072250 msgr2=0x7f51c0070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51c5cad640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c0072250 0x7f51c0070510 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009e50 tx=0x7f51b802bed0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.750+0000 7f51c5cad640 1 -- 172.21.15.111:0/1279103400 shutdown_connections 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.751+0000 7f51c5cad640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c0072250 0x7f51c0070510 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.751+0000 7f51c5cad640 1 --2- 172.21.15.111:0/1279103400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c00b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.751+0000 7f51c5cad640 1 -- 172.21.15.111:0/1279103400 >> 172.21.15.111:0/1279103400 conn(0x7f51c00b0cf0 msgr2=0x7f51c00b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.751+0000 7f51c5cad640 1 -- 172.21.15.111:0/1279103400 shutdown_connections 2023-12-04T05:41:06.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.751+0000 7f51c5cad640 1 -- 172.21.15.111:0/1279103400 wait complete. 2023-12-04T05:41:06.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.752+0000 7f51c5cad640 1 Processor -- start 2023-12-04T05:41:06.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.752+0000 7f51c5cad640 1 -- start start 2023-12-04T05:41:06.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.752+0000 7f51c5cad640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51c0072250 0x7f51c013d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 0x7f51c01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0155670 con 0x7f51c015c770 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0155110 con 0x7f51c0072250 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0155390 con 0x7f51c00b4300 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 0x7f51c01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 0x7f51c01506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57930/0 (socket says 172.21.15.111:57930) 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 -- 172.21.15.111:0/2408852258 learned_addr learned my addr 172.21.15.111:0/2408852258 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bf7fe640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51c0072250 0x7f51c013d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51beffd640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 -- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 msgr2=0x7f51c0154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c0154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 -- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51c0072250 msgr2=0x7f51c013d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51c0072250 0x7f51c013d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 -- 172.21.15.111:0/2408852258 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8009b30 con 0x7f51c015c770 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51bffff640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 0x7f51c01506a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f51b8036040 tx=0x7f51b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b803e450 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b803e5d0 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b803e980 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c013a710 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.753+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c013acd0 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.755+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c013a570 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.755+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f51b803eae0 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.755+0000 7f51a7fff640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f51ac06b2a0 0x7f51ac06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.755+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51b80ac940 con 0x7f51c015c770 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.755+0000 7f51bf7fe640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f51ac06b2a0 0x7f51ac06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:06.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.756+0000 7f51bf7fe640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f51ac06b2a0 0x7f51ac06d760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ab60 tx=0x7f51b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:06.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.759+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51c013a570 con 0x7f51c015c770 2023-12-04T05:41:06.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.994+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f51c013a570 con 0x7f51c015c770 2023-12-04T05:41:06.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:06.997+0000 7f51a7fff640 1 -- 172.21.15.111:0/2408852258 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f51b80430b0 con 0x7f51c015c770 2023-12-04T05:41:06.998 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2023-12-04T05:41:06.999 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQDLZW1l+5T3ABAA7h+0Ksz9xnrfFZKgsBbhIA== 2023-12-04T05:41:06.999 INFO:teuthology.orchestra.run.smithi111.stdout: caps mds = "allow" 2023-12-04T05:41:06.999 INFO:teuthology.orchestra.run.smithi111.stdout: caps mon = "allow r" 2023-12-04T05:41:06.999 INFO:teuthology.orchestra.run.smithi111.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-04T05:41:06.999 INFO:teuthology.orchestra.run.smithi111.stderr:updated caps for client.0 2023-12-04T05:41:07.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f51ac06b2a0 msgr2=0x7f51ac06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f51ac06b2a0 0x7f51ac06d760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ab60 tx=0x7f51b4008040 comp rx=0 tx=0).stop 2023-12-04T05:41:07.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 msgr2=0x7f51c01506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 0x7f51c01506a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f51b8036040 tx=0x7f51b8004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 shutdown_connections 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f51ac06b2a0 0x7f51ac06d760 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51c015c770 0x7f51c01506a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51c00b4300 0x7f51c0154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 --2- 172.21.15.111:0/2408852258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51c0072250 0x7f51c013d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 >> 172.21.15.111:0/2408852258 conn(0x7f51c00b0cf0 msgr2=0x7f51c0071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 shutdown_connections 2023-12-04T05:41:07.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.000+0000 7f51c5cad640 1 -- 172.21.15.111:0/2408852258 wait complete. 2023-12-04T05:41:07.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:41:07.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.110+0000 7fe7f86ee640 1 Processor -- start 2023-12-04T05:41:07.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.110+0000 7fe7f86ee640 1 -- start start 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f86ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe7f006cae0 0x7fe7f006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f86ee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe7f00b3310 0x7fe7f0159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f86ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f0154220 con 0x7fe7f006c250 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f86ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f015d2f0 con 0x7fe7f006cae0 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f86ee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f006d400 con 0x7fe7f00b3310 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f648a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe7f006cae0 0x7fe7f006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f648a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe7f006cae0 0x7fe7f006cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57946/0 (socket says 172.21.15.111:57946) 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f648a640 1 -- 172.21.15.111:0/1867432320 learned_addr learned my addr 172.21.15.111:0/1867432320 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f6c8b640 1 --1- 172.21.15.111:0/1867432320 >> v1:172.21.15.111:6789/0 conn(0x7fe7f006c250 0x7fe7f015dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39616/0 (socket says 172.21.15.111:39616) 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f5c89640 1 --2- 172.21.15.111:0/1867432320 >> v2:172.21.15.192:3300/0 conn(0x7fe7f00b3310 0x7fe7f0159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f648a640 1 -- 172.21.15.111:0/1867432320 >> v2:172.21.15.192:3300/0 conn(0x7fe7f00b3310 msgr2=0x7fe7f0159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f648a640 1 --2- 172.21.15.111:0/1867432320 >> v2:172.21.15.192:3300/0 conn(0x7fe7f00b3310 0x7fe7f0159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.111+0000 7fe7f648a640 1 -- 172.21.15.111:0/1867432320 >> v1:172.21.15.111:6789/0 conn(0x7fe7f006c250 legacy=0x7fe7f015dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.112+0000 7fe7f5488640 1 -- 172.21.15.111:0/1867432320 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798259783 0 0) 0x7fe7e8001110 con 0x7fe7f006c250 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.112+0000 7fe7f648a640 1 -- 172.21.15.111:0/1867432320 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f015a5c0 con 0x7fe7f006cae0 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.112+0000 7fe7f648a640 1 --2- 172.21.15.111:0/1867432320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006cae0 0x7fe7f006cec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00718b0 tx=0x7fe7ec02e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6469f53d3f5d909 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.112+0000 7fe7f5488640 1 -- 172.21.15.111:0/1867432320 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec00b0d0 con 0x7fe7f006cae0 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.112+0000 7fe7f5488640 1 -- 172.21.15.111:0/1867432320 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec00b230 con 0x7fe7f006cae0 2023-12-04T05:41:07.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.112+0000 7fe7f5488640 1 -- 172.21.15.111:0/1867432320 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec009600 con 0x7fe7f006cae0 2023-12-04T05:41:07.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 -- 172.21.15.111:0/1867432320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006cae0 msgr2=0x7fe7f006cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/1867432320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006cae0 0x7fe7f006cec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00718b0 tx=0x7fe7ec02e180 comp rx=0 tx=0).stop 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 -- 172.21.15.111:0/1867432320 shutdown_connections 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/1867432320 >> v2:172.21.15.192:3300/0 conn(0x7fe7f00b3310 0x7fe7f0159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/1867432320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006cae0 0x7fe7f006cec0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 -- 172.21.15.111:0/1867432320 >> 172.21.15.111:0/1867432320 conn(0x7fe7f006a4c0 msgr2=0x7fe7f006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 -- 172.21.15.111:0/1867432320 shutdown_connections 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.113+0000 7fe7f86ee640 1 -- 172.21.15.111:0/1867432320 wait complete. 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.114+0000 7fe7f86ee640 1 Processor -- start 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.114+0000 7fe7f86ee640 1 -- start start 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f86ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 0x7fe7f014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f86ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7f006cae0 0x7fe7f0150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f86ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7f00b3310 0x7fe7f0154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f86ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0150620 con 0x7fe7f006c250 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f86ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f014b9e0 con 0x7fe7f00b3310 2023-12-04T05:41:07.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f86ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f014fa10 con 0x7fe7f006cae0 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 0x7fe7f014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 0x7fe7f014b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57950/0 (socket says 172.21.15.111:57950) 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 -- 172.21.15.111:0/2786116216 learned_addr learned my addr 172.21.15.111:0/2786116216 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f6c8b640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7f00b3310 0x7fe7f0154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f5c89640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7f006cae0 0x7fe7f0150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 -- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7f006cae0 msgr2=0x7fe7f0150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7f006cae0 0x7fe7f0150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 -- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7f00b3310 msgr2=0x7fe7f0154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7f00b3310 0x7fe7f0154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 -- 172.21.15.111:0/2786116216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec00bd40 con 0x7fe7f006c250 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f6c8b640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7f00b3310 0x7fe7f0154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.115+0000 7fe7f648a640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 0x7fe7f014b4a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000f80 tx=0x7fe7ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.116+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec00b0d0 con 0x7fe7f006c250 2023-12-04T05:41:07.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.116+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec00b720 con 0x7fe7f006c250 2023-12-04T05:41:07.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.116+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f014fcf0 con 0x7fe7f006c250 2023-12-04T05:41:07.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.116+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec040880 con 0x7fe7f006c250 2023-12-04T05:41:07.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.116+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f01502b0 con 0x7fe7f006c250 2023-12-04T05:41:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.117+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f00693d0 con 0x7fe7f006c250 2023-12-04T05:41:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.117+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe7ec0409e0 con 0x7fe7f006c250 2023-12-04T05:41:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.117+0000 7fe7e6ffd640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7d406b2d0 0x7fe7d406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.117+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec0aec90 con 0x7fe7f006c250 2023-12-04T05:41:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.118+0000 7fe7f5c89640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7d406b2d0 0x7fe7d406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.118+0000 7fe7f5c89640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7d406b2d0 0x7fe7d406d790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0157110 tx=0x7fe7e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.122+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe7ec079b00 con 0x7fe7f006c250 2023-12-04T05:41:07.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.325+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7f0071b80 con 0x7fe7f006c250 2023-12-04T05:41:07.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.325+0000 7fe7e6ffd640 1 -- 172.21.15.111:0/2786116216 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fe7ec07d120 con 0x7fe7f006c250 2023-12-04T05:41:07.327 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:07.327 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/956157809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":956157809},{"type":"v1","addr":"172.21.15.192:6835","nonce":956157809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6836/619465105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":619465105},{"type":"v1","addr":"172.21.15.111:6836","nonce":619465105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:41:04.639249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:41:07.327 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 20 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7d406b2d0 msgr2=0x7fe7d406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7d406b2d0 0x7fe7d406d790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0157110 tx=0x7fe7e0008040 comp rx=0 tx=0).stop 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 msgr2=0x7fe7f014b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 0x7fe7f014b4a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000f80 tx=0x7fe7ec004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 shutdown_connections 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7d406b2d0 0x7fe7d406d790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7f00b3310 0x7fe7f0154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7f006cae0 0x7fe7f0150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 --2- 172.21.15.111:0/2786116216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f006c250 0x7fe7f014b4a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 >> 172.21.15.111:0/2786116216 conn(0x7fe7f006a4c0 msgr2=0x7fe7f00cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 shutdown_connections 2023-12-04T05:41:07.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.327+0000 7fe7f86ee640 1 -- 172.21.15.111:0/2786116216 wait complete. 2023-12-04T05:41:07.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-04T05:41:02.644246+0000', 'modified': '2023-12-04T05:41:04.639249+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5868}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5868': {'gid': 5868, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6837/2442390644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6835', 'nonce': 2442390644}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 2442390644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-04T05:41:07.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-04T05:41:07.743 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-04T05:41:07.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.842+0000 7f398781c640 1 Processor -- start 2023-12-04T05:41:07.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.843+0000 7f398781c640 1 -- start start 2023-12-04T05:41:07.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.843+0000 7f398781c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.843+0000 7f398781c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39800b20a0 0x7f39800d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.843+0000 7f398781c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39800cdcb0 con 0x7f39800b2ad0 2023-12-04T05:41:07.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.843+0000 7f398781c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39800d20a0 con 0x7f39800b20a0 2023-12-04T05:41:07.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.843+0000 7f398781c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39800d39e0 con 0x7f39800d1780 2023-12-04T05:41:07.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39800b20a0 0x7f39800d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39800b20a0 0x7f39800d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44370/0 (socket says 172.21.15.111:44370) 2023-12-04T05:41:07.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 -- 172.21.15.111:0/1761378698 learned_addr learned my addr 172.21.15.111:0/1761378698 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f39855b8640 1 --2- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 -- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 msgr2=0x7f39800d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 --2- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 -- 172.21.15.111:0/1761378698 >> v1:172.21.15.192:6789/0 conn(0x7f39800b2ad0 legacy=0x7f39800ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.844+0000 7f3984db7640 1 -- 172.21.15.111:0/1761378698 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800d3cc0 con 0x7f39800b20a0 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.845+0000 7f3984db7640 1 --2- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800d1b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f397c009870 tx=0x7f397c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c29b3249abb2d3e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.845+0000 7f3973fff640 1 -- 172.21.15.111:0/1761378698 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c031640 con 0x7f39800b20a0 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.845+0000 7f3973fff640 1 -- 172.21.15.111:0/1761378698 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0317a0 con 0x7f39800b20a0 2023-12-04T05:41:07.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.845+0000 7f3973fff640 1 -- 172.21.15.111:0/1761378698 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c031b30 con 0x7f39800b20a0 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 -- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 msgr2=0x7f39800d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 --2- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800d1b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f397c009870 tx=0x7f397c02b8f0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 -- 172.21.15.111:0/1761378698 shutdown_connections 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 --2- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800d1b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 --2- 172.21.15.111:0/1761378698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 -- 172.21.15.111:0/1761378698 >> 172.21.15.111:0/1761378698 conn(0x7f398009c4e0 msgr2=0x7f39800d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 -- 172.21.15.111:0/1761378698 shutdown_connections 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.846+0000 7f398781c640 1 -- 172.21.15.111:0/1761378698 wait complete. 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.847+0000 7f398781c640 1 Processor -- start 2023-12-04T05:41:07.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.847+0000 7f398781c640 1 -- start start 2023-12-04T05:41:07.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f398781c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f398781c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 0x7f39800b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f398781c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f398781c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39800cc3d0 con 0x7f39800b2ad0 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f398781c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39800b8b40 con 0x7f39800b20a0 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f398781c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398015f5c0 con 0x7f39800d1780 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 0x7f39800b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 0x7f39800b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57964/0 (socket says 172.21.15.111:57964) 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 -- 172.21.15.111:0/688313341 learned_addr learned my addr 172.21.15.111:0/688313341 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f39855b8640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3985db9640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 -- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 msgr2=0x7f39800cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 -- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 msgr2=0x7f39800b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3984db7640 1 -- 172.21.15.111:0/688313341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0094d0 con 0x7f39800b2ad0 2023-12-04T05:41:07.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.848+0000 7f3985db9640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.849+0000 7f3984db7640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 0x7f39800b5dc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f397c000c00 tx=0x7f397c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.849+0000 7f3971ffb640 1 -- 172.21.15.111:0/688313341 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c03a930 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.849+0000 7f3971ffb640 1 -- 172.21.15.111:0/688313341 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c03aa90 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.849+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398015f8a0 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.849+0000 7f3971ffb640 1 -- 172.21.15.111:0/688313341 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c03ae40 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.849+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398015f2b0 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.850+0000 7f3971ffb640 1 -- 172.21.15.111:0/688313341 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f397c02f050 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.851+0000 7f3971ffb640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f396406b2a0 0x7f396406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.851+0000 7f3971ffb640 1 -- 172.21.15.111:0/688313341 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f397c0ab8b0 con 0x7f39800b2ad0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.851+0000 7f39855b8640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f396406b2a0 0x7f396406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.851+0000 7f39855b8640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f396406b2a0 0x7f396406d760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39800b8a70 tx=0x7f3974009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 Processor -- start 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 -- start start 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b67fa0 0x55fb50b3af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b6ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b3b480 0x55fb50b6b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50afbd10 con 0x55fb50b68380 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50aadb90 con 0x55fb50b67fa0 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.852+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb50aab280 con 0x55fb50b3b480 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b6ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b6ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57970/0 (socket says 172.21.15.111:57970) 2023-12-04T05:41:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 -- 172.21.15.111:0/2466518502 learned_addr learned my addr 172.21.15.111:0/2466518502 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f39855b8640 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b67fa0 0x55fb50b3af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3985db9640 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b3b480 0x55fb50b6b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 -- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b3b480 msgr2=0x55fb50b6b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b3b480 0x55fb50b6b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 -- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b67fa0 msgr2=0x55fb50b3af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b67fa0 0x55fb50b3af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 -- 172.21.15.111:0/2466518502 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb50a6c500 con 0x55fb50b68380 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3985db9640 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b3b480 0x55fb50b6b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f3984db7640 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b6ad30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f397c0768e0 tx=0x7f397c07ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84c8fa6d3a77a0b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f396affd640 1 -- 172.21.15.111:0/2466518502 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c08f070 con 0x55fb50b68380 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f396affd640 1 -- 172.21.15.111:0/2466518502 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0826e0 con 0x55fb50b68380 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.853+0000 7f396affd640 1 -- 172.21.15.111:0/2466518502 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c078780 con 0x55fb50b68380 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 -- 172.21.15.111:0/2466518502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 msgr2=0x55fb50b6ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b6ad30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f397c0768e0 tx=0x7f397c07ac20 comp rx=0 tx=0).stop 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 -- 172.21.15.111:0/2466518502 shutdown_connections 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b3b480 0x55fb50b6b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b6ad30 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2466518502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b67fa0 0x55fb50b3af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 -- 172.21.15.111:0/2466518502 >> 172.21.15.111:0/2466518502 conn(0x55fb50b3d1b0 msgr2=0x55fb50b85d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 -- 172.21.15.111:0/2466518502 shutdown_connections 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 -- 172.21.15.111:0/2466518502 wait complete. 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 -- start start 2023-12-04T05:41:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.854+0000 7f398a3b3740 1 Processor -- start 2023-12-04T05:41:07.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b80f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b86230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50b66d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50aa6b70 con 0x55fb50b668a0 2023-12-04T05:41:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50a60e00 con 0x55fb50b68380 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb50a60aa0 con 0x55fb50b67fa0 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f398a3b3740 10 client.0 _ll_register_callbacks cb 0x55fb50a652d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f3985db9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50b66d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f3985db9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50b66d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57980/0 (socket says 172.21.15.111:57980) 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f3985db9640 1 -- 172.21.15.111:0/3551711621 learned_addr learned my addr 172.21.15.111:0/3551711621 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f39855b8640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b80f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.855+0000 7f3984db7640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b86230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f3985db9640 1 -- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 msgr2=0x55fb50b80f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f3985db9640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b80f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f3985db9640 1 -- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 msgr2=0x55fb50b86230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f3985db9640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b86230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f3985db9640 1 -- 172.21.15.111:0/3551711621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398015f2b0 con 0x55fb50b668a0 2023-12-04T05:41:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f3985db9640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50b66d20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f397800f4b0 tx=0x7f397800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b21d6144854cd52 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f395ffff640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f395ffff640 1 -- 172.21.15.111:0/3551711621 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978003b80 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f395ffff640 1 -- 172.21.15.111:0/3551711621 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3978003ce0 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb50a9fb50 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f395ffff640 1 -- 172.21.15.111:0/3551711621 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3978003ea0 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 10 client.6000 fetch_fsmap learned FSMap version 20 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55fb50aa51f0 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f395ffff640 1 -- 172.21.15.111:0/3551711621 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb50a9fb50 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f395ffff640 1 -- 172.21.15.111:0/3551711621 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3978012070 con 0x55fb50b668a0 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 10 client.6000 fetch_fsmap finished waiting for FSMap version 20 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 10 client.6000 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 20 client.6000 populate_metadata read hostname 'smithi111' 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50b7edd0 0x55fb50b99400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 4 client.6000 mds_command: new command op to 5868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.856+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 --> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb50a9f870 con 0x55fb50b7edd0 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f39855b8640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50b7edd0 0x55fb50b99400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f395ffff640 10 client.6000 ms_handle_connect on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f39855b8640 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50b7edd0 0x55fb50b99400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c14981907eec2a server_cookie=f958894f61df3162 in_seq=0 out_seq=0 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f395ffff640 1 -- 172.21.15.111:0/3551711621 <== mds.0 v2:172.21.15.111:6835/2442390644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb50a9f870 con 0x55fb50b7edd0 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f395ffff640 10 client.6000 handle_command_reply: tid=0 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f398a3b3740 1 client.6000 shutdown 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 msgr2=0x55fb50b66d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50b66d20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f397800f4b0 tx=0x7f397800f8b0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 shutdown_connections 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50b7edd0 0x55fb50b99400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.857+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50b66d20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b86230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3551711621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b80f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f395ffff640 0 client.6000 ms_handle_reset on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 >> 172.21.15.111:0/3551711621 conn(0x55fb50b3b480 msgr2=0x55fb50b6afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 shutdown_connections 2023-12-04T05:41:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f398a3b3740 1 -- 172.21.15.111:0/3551711621 wait complete. 2023-12-04T05:41:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.858+0000 7f398a3b3740 20 client.6000 trim_cache size 0 max 16384 2023-12-04T05:41:07.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.873+0000 7f398a3b3740 1 Processor -- start 2023-12-04T05:41:07.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.873+0000 7f398a3b3740 1 -- start start 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b668a0 0x55fb50b98420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f398a3b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50a26270 con 0x55fb50b68380 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50b367f0 con 0x55fb50b668a0 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb50b6b930 con 0x55fb50b67fa0 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57992/0 (socket says 172.21.15.111:57992) 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 -- 172.21.15.111:0/3093045378 learned_addr learned my addr 172.21.15.111:0/3093045378 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 -- 172.21.15.111:0/3093045378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 msgr2=0x55fb50b9f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 -- 172.21.15.111:0/3093045378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b668a0 msgr2=0x55fb50b98420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b668a0 0x55fb50b98420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 -- 172.21.15.111:0/3093045378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb50a42d00 con 0x55fb50b68380 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.874+0000 7f3985db9640 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b89240 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3978007f10 tx=0x7f397800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a0f902220e2b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f396b7fe640 1 -- 172.21.15.111:0/3093045378 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978003d80 con 0x55fb50b68380 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f396b7fe640 1 -- 172.21.15.111:0/3093045378 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397800d050 con 0x55fb50b68380 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f396b7fe640 1 -- 172.21.15.111:0/3093045378 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978006d50 con 0x55fb50b68380 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 -- 172.21.15.111:0/3093045378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 msgr2=0x55fb50b89240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b89240 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3978007f10 tx=0x7f397800a000 comp rx=0 tx=0).stop 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 -- 172.21.15.111:0/3093045378 shutdown_connections 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b9f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 --2- 172.21.15.111:0/3093045378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b668a0 0x55fb50b98420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 -- 172.21.15.111:0/3093045378 >> 172.21.15.111:0/3093045378 conn(0x55fb50b3b480 msgr2=0x55fb50b41100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 -- 172.21.15.111:0/3093045378 shutdown_connections 2023-12-04T05:41:07.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.875+0000 7f398a3b3740 1 -- 172.21.15.111:0/3093045378 wait complete. 2023-12-04T05:41:07.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.876+0000 7f398a3b3740 1 -- start start 2023-12-04T05:41:07.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.876+0000 7f398a3b3740 1 Processor -- start 2023-12-04T05:41:07.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50beda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50b7c570 con 0x55fb50b668a0 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb50bf1220 con 0x55fb50b68380 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb50b7c320 con 0x55fb50b67fa0 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 10 client.0 _ll_register_callbacks cb 0x7f3978002120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50beda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50beda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57994/0 (socket says 172.21.15.111:57994) 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 -- 172.21.15.111:0/2305256071 learned_addr learned my addr 172.21.15.111:0/2305256071 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f3984db7640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f3985db9640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 -- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 msgr2=0x55fb50b86770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b86770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 -- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 msgr2=0x55fb50b89240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 -- 172.21.15.111:0/2305256071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb50aa51f0 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f39855b8640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50beda10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f39740092d0 tx=0x7f39740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f508d20ed80e32d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f395ffff640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f395ffff640 1 -- 172.21.15.111:0/2305256071 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397401c040 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f395ffff640 1 -- 172.21.15.111:0/2305256071 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f397402a320 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f397c0826e0 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f395ffff640 1 -- 172.21.15.111:0/2305256071 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f397402a4e0 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 10 client.6006 fetch_fsmap learned FSMap version 20 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.877+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55fb50a684f0 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f395ffff640 1 -- 172.21.15.111:0/2305256071 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0826e0 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f395ffff640 1 -- 172.21.15.111:0/2305256071 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f397402a640 con 0x55fb50b668a0 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 10 client.6006 fetch_fsmap finished waiting for FSMap version 20 2023-12-04T05:41:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 10 client.6006 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 20 client.6006 populate_metadata read hostname 'smithi111' 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50bf0510 0x55fb50bf08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 4 client.6006 mds_command: new command op to 5868 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 --> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb507a76c0 con 0x55fb50bf0510 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f3984db7640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50bf0510 0x55fb50bf08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f395ffff640 10 client.6006 ms_handle_connect on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f3984db7640 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50bf0510 0x55fb50bf08f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0de66d76f1026f server_cookie=207cb95736e5bcc5 in_seq=0 out_seq=0 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f395ffff640 1 -- 172.21.15.111:0/2305256071 <== mds.0 v2:172.21.15.111:6835/2442390644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fb507a76c0 con 0x55fb50bf0510 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f395ffff640 10 client.6006 handle_command_reply: tid=0 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 client.6006 shutdown 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 msgr2=0x55fb50beda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50beda10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f39740092d0 tx=0x7f39740046c0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 shutdown_connections 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55fb50bf0510 0x55fb50bf08f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb50b68380 0x55fb50b89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fb50b67fa0 0x55fb50b86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 --2- 172.21.15.111:0/2305256071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fb50b668a0 0x55fb50beda10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 >> 172.21.15.111:0/2305256071 conn(0x55fb50b3b480 msgr2=0x55fb50b41100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.878+0000 7f395ffff640 0 client.6006 ms_handle_reset on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 shutdown_connections 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398a3b3740 1 -- 172.21.15.111:0/2305256071 wait complete. 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398a3b3740 20 client.6006 trim_cache size 0 max 16384 2023-12-04T05:41:07.887 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:41:07.888 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "9686316e-a01b-4981-b0b7-e21bdb185d5e", 2023-12-04T05:41:07.888 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-12-04T05:41:07.888 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5868, 2023-12-04T05:41:07.888 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-12-04T05:41:07.888 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-12-04T05:41:07.888 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-12-04T05:41:07.889 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 4.5676290110000002, 2023-12-04T05:41:07.889 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 20, 2023-12-04T05:41:07.889 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 45, 2023-12-04T05:41:07.889 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 45, 2023-12-04T05:41:07.889 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 9.6268269400000008 2023-12-04T05:41:07.889 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:41:07.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f396406b2a0 msgr2=0x7f396406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f396406b2a0 0x7f396406d760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39800b8a70 tx=0x7f3974009190 comp rx=0 tx=0).stop 2023-12-04T05:41:07.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 msgr2=0x7f39800b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 0x7f39800b5dc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f397c000c00 tx=0x7f397c0046c0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 shutdown_connections 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f396406b2a0 0x7f396406d760 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39800d1780 0x7f39800cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39800b2ad0 0x7f39800b5dc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 --2- 172.21.15.111:0/688313341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39800b20a0 0x7f39800b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 >> 172.21.15.111:0/688313341 conn(0x7f398009c4e0 msgr2=0x7f39800b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 shutdown_connections 2023-12-04T05:41:07.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.879+0000 7f398781c640 1 -- 172.21.15.111:0/688313341 wait complete. 2023-12-04T05:41:07.892 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:41:07.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.990+0000 7f6c2f948640 1 Processor -- start 2023-12-04T05:41:07.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.990+0000 7f6c2f948640 1 -- start start 2023-12-04T05:41:07.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2f948640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6c2806c960 0x7f6c2815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2f948640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c280aea20 con 0x7f6c28140200 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2f948640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2813d2c0 con 0x7f6c2813cee0 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2f948640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2815b3a0 con 0x7f6c2806c960 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2cee3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6c2806c960 0x7f6c2815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2dee5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6c28140200 0x7f6c280cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39624/0 (socket says 172.21.15.111:39624) 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2dee5640 1 -- 172.21.15.111:0/3736701543 learned_addr learned my addr 172.21.15.111:0/3736701543 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2cee3640 1 -- 172.21.15.111:0/3736701543 >> v1:172.21.15.192:6789/0 conn(0x7f6c2813cee0 legacy=0x7f6c2806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2cee3640 1 -- 172.21.15.111:0/3736701543 >> v1:172.21.15.111:6789/0 conn(0x7f6c28140200 legacy=0x7f6c280cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.991+0000 7f6c2cee3640 1 -- 172.21.15.111:0/3736701543 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2815b0c0 con 0x7f6c2806c960 2023-12-04T05:41:07.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.992+0000 7f6c2cee3640 1 --2- 172.21.15.111:0/3736701543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2806c960 0x7f6c2815a490 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6c180097a0 tx=0x7f6c1802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341d6cdc1f5e424 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.992+0000 7f6c1ffff640 1 -- 172.21.15.111:0/3736701543 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1802e070 con 0x7f6c2806c960 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.992+0000 7f6c1ffff640 1 -- 172.21.15.111:0/3736701543 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c180315a0 con 0x7f6c2806c960 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.992+0000 7f6c1ffff640 1 -- 172.21.15.111:0/3736701543 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18031930 con 0x7f6c2806c960 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 -- 172.21.15.111:0/3736701543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2806c960 msgr2=0x7f6c2815a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 --2- 172.21.15.111:0/3736701543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2806c960 0x7f6c2815a490 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6c180097a0 tx=0x7f6c1802b840 comp rx=0 tx=0).stop 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 -- 172.21.15.111:0/3736701543 shutdown_connections 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 --2- 172.21.15.111:0/3736701543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2806c960 0x7f6c2815a490 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 -- 172.21.15.111:0/3736701543 >> 172.21.15.111:0/3736701543 conn(0x7f6c2806a4c0 msgr2=0x7f6c2806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 -- 172.21.15.111:0/3736701543 shutdown_connections 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.993+0000 7f6c2f948640 1 -- 172.21.15.111:0/3736701543 wait complete. 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.994+0000 7f6c2f948640 1 Processor -- start 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.994+0000 7f6c2f948640 1 -- start start 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c2806c960 0x7f6c281364e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 0x7f6c2813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c28140200 0x7f6c281380d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2813eaf0 con 0x7f6c2813cee0 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2813dee0 con 0x7f6c28140200 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2813e1c0 con 0x7f6c2806c960 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 0x7f6c2813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 0x7f6c2813d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58020/0 (socket says 172.21.15.111:58020) 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 -- 172.21.15.111:0/2696182155 learned_addr learned my addr 172.21.15.111:0/2696182155 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2dee5640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c28140200 0x7f6c281380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2d6e4640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c2806c960 0x7f6c281364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 -- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c2806c960 msgr2=0x7f6c281364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c2806c960 0x7f6c281364e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 -- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c28140200 msgr2=0x7f6c281380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c28140200 0x7f6c281380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 -- 172.21.15.111:0/2696182155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18009400 con 0x7f6c2813cee0 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2cee3640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 0x7f6c2813d460 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6c18034040 tx=0x7f6c18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:07.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1803cb90 con 0x7f6c2813cee0 2023-12-04T05:41:08.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1803cd10 con 0x7f6c2813cee0 2023-12-04T05:41:08.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2813e4a0 con 0x7f6c2813cee0 2023-12-04T05:41:08.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.995+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18032670 con 0x7f6c2813cee0 2023-12-04T05:41:08.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.996+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2813c1e0 con 0x7f6c2813cee0 2023-12-04T05:41:08.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.997+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c281508d0 con 0x7f6c2813cee0 2023-12-04T05:41:08.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.997+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6c180327d0 con 0x7f6c2813cee0 2023-12-04T05:41:08.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.997+0000 7f6c1dffb640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6c0c06b2a0 0x7f6c0c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:08.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.997+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c180ad760 con 0x7f6c2813cee0 2023-12-04T05:41:08.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.997+0000 7f6c2d6e4640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6c0c06b2a0 0x7f6c0c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:08.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:07.998+0000 7f6c2d6e4640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6c0c06b2a0 0x7f6c0c06d760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c281393e0 tx=0x7f6c24007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:08.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.002+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c18078680 con 0x7f6c2813cee0 2023-12-04T05:41:08.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.205+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c2813d2c0 con 0x7f6c2813cee0 2023-12-04T05:41:08.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.205+0000 7f6c1dffb640 1 -- 172.21.15.111:0/2696182155 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c2813d2c0 con 0x7f6c2813cee0 2023-12-04T05:41:08.207 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:41:08.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6c0c06b2a0 msgr2=0x7f6c0c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6c0c06b2a0 0x7f6c0c06d760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c281393e0 tx=0x7f6c24007e30 comp rx=0 tx=0).stop 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 msgr2=0x7f6c2813d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 0x7f6c2813d460 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6c18034040 tx=0x7f6c18004730 comp rx=0 tx=0).stop 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 shutdown_connections 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6c0c06b2a0 0x7f6c0c06d760 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c28140200 0x7f6c281380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c2813cee0 0x7f6c2813d460 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 --2- 172.21.15.111:0/2696182155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c2806c960 0x7f6c281364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 >> 172.21.15.111:0/2696182155 conn(0x7f6c2806a4c0 msgr2=0x7f6c2815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 shutdown_connections 2023-12-04T05:41:08.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:08.207+0000 7f6c2f948640 1 -- 172.21.15.111:0/2696182155 wait complete. 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:41:08.219 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:41:08.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.220 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-12-04T05:41:08.235 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:41:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:41:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:41:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-12-04T05:41:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42049sec preferred_lft 42049sec 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-12-04T05:41:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4e:91:07:f7:96:4c brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::142c:d5ff:fef0:ffc5/64 scope link 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8e:82:5e:fe:4f:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::8c82:5eff:fefe:4fbf/64 scope link 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 4e:91:07:f7:96:4c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::4c91:7ff:fef7:964c/64 scope link 2023-12-04T05:41:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:08.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.238 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:41:08.301 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-12-04T05:41:08.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.303 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-12-04T05:41:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-12-04T05:41:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:41:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:41:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:41:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:41:08.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.360 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-12-04T05:41:08.360 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-12-04T05:41:08.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:41:08.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:41:08.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:41:08.360 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-12-04T05:41:08.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.454 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:41:08.469 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-04T05:41:08.469 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-04T05:41:08.469 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:41:08.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-12-04T05:41:08.536 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-04T05:41:08.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:41:08.536 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-04T05:41:08.575 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:41:08.573+0000 7f344645d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-04T05:41:08.589 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-04T05:41:08.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:41:08.589 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:41:08.643 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:41:08.713 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:41:08.769 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:41:08.769 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:41:08.769 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:41:08.769 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:41:08.769 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:41:08.769 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:41:08.770 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:41:08.770 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:41:08.770 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:41:08.770 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:41:08.770 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:41:08.770 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:41:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:41:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:41:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:41:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:41:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:41:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:41:08.772 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:41:08.772 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:41:08.772 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:41:08.772 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:41:08.772 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:41:08.772 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:41:08.773 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:41:08.773 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:41:08.773 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:41:08.773 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:41:08.773 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:41:08.773 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:41:08.774 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:41:08.774 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:41:08.774 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:41:08.774 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:41:08.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.775 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:41:08.823 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:41:08.824 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-04T05:41:08.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-12-04T05:41:08.892 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:41:08.947 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2023-12-04T05:41:08.948 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2023-12-04T05:41:08.949 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:41:08.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:08.950 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-12-04T05:41:08.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2023-12-04T05:41:08.949+0000 7f2dc9e3b480 -1 init, newargv = 0x564a1d97abc0 newargc=15 2023-12-04T05:41:08.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[108271]: starting ceph client 2023-12-04T05:41:08.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[108271]: starting fuse 2023-12-04T05:41:08.965 INFO:teuthology.orchestra.run.smithi111.stdout:41 2023-12-04T05:41:08.966 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-04T05:41:08.966 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-12-04T05:41:08.967 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:41:08.968 DEBUG:teuthology.orchestra.run.smithi111:> 2023-12-04T05:41:08.968 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2023-12-04T05:41:08.968 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-12-04T05:41:09.086 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.108271.asok 2023-12-04T05:41:09.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:41:09.096 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.108271.asok 2023-12-04T05:41:09.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.108271.asok status 2023-12-04T05:41:09.232 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:41:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-12-04T05:41:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:41:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "108271", 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 6024, 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 6024 2023-12-04T05:41:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2204094072 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2204094072 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.6024 192.168.0.1:0/2204094072", 2023-12-04T05:41:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/2204094072", 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 20, 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 45, 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-12-04T05:41:09.235 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:41:09.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.241 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-12-04T05:41:09.241 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-04T05:41:09.256 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-12-04T05:41:09.256 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-04T05:41:09.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.256 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-04T05:41:09.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-04T05:41:09.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.462+0000 7fea87a94640 1 Processor -- start 2023-12-04T05:41:09.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.463+0000 7fea87a94640 1 -- start start 2023-12-04T05:41:09.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.463+0000 7fea87a94640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea800d01e0 0x7fea800ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:09.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.463+0000 7fea87a94640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea800cdcb0 con 0x7fea800b20a0 2023-12-04T05:41:09.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.463+0000 7fea87a94640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea800d39e0 con 0x7fea800d1780 2023-12-04T05:41:09.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.463+0000 7fea87a94640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea800d05f0 con 0x7fea800d01e0 2023-12-04T05:41:09.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.464+0000 7fea86031640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea800d01e0 0x7fea800ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:09.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.464+0000 7fea8502f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fea800b20a0 0x7fea800b2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39636/0 (socket says 172.21.15.111:39636) 2023-12-04T05:41:09.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.464+0000 7fea8502f640 1 -- 172.21.15.111:0/3294551959 learned_addr learned my addr 172.21.15.111:0/3294551959 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:09.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.464+0000 7fea86031640 1 -- 172.21.15.111:0/3294551959 >> v1:172.21.15.192:6789/0 conn(0x7fea800d1780 legacy=0x7fea800d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:09.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.464+0000 7fea86031640 1 -- 172.21.15.111:0/3294551959 >> v1:172.21.15.111:6789/0 conn(0x7fea800b20a0 legacy=0x7fea800b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:09.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.464+0000 7fea86031640 1 -- 172.21.15.111:0/3294551959 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800d3cc0 con 0x7fea800d01e0 2023-12-04T05:41:09.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea86031640 1 --2- 172.21.15.111:0/3294551959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea800ce750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fea78009d90 tx=0x7fea7802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93217050a45f7c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:09.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea8482e640 1 -- 172.21.15.111:0/3294551959 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7802cbd0 con 0x7fea800d01e0 2023-12-04T05:41:09.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea8482e640 1 -- 172.21.15.111:0/3294551959 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7802cd30 con 0x7fea800d01e0 2023-12-04T05:41:09.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea8482e640 1 -- 172.21.15.111:0/3294551959 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7802cea0 con 0x7fea800d01e0 2023-12-04T05:41:09.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 -- 172.21.15.111:0/3294551959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 msgr2=0x7fea800ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:09.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 --2- 172.21.15.111:0/3294551959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea800ce750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fea78009d90 tx=0x7fea7802be20 comp rx=0 tx=0).stop 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 -- 172.21.15.111:0/3294551959 shutdown_connections 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 --2- 172.21.15.111:0/3294551959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea800ce750 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 -- 172.21.15.111:0/3294551959 >> 172.21.15.111:0/3294551959 conn(0x7fea8009c4e0 msgr2=0x7fea800d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 -- 172.21.15.111:0/3294551959 shutdown_connections 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.465+0000 7fea87a94640 1 -- 172.21.15.111:0/3294551959 wait complete. 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.466+0000 7fea87a94640 1 Processor -- start 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.466+0000 7fea87a94640 1 -- start start 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea87a94640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea800b20a0 0x7fea800ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea87a94640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea8015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea87a94640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800d1780 0x7fea800b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea87a94640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800ca870 con 0x7fea800d01e0 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea87a94640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800cab50 con 0x7fea800d1780 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea87a94640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800cae30 con 0x7fea800b20a0 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea8502f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea8015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:09.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea8502f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea8015a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58044/0 (socket says 172.21.15.111:58044) 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea8502f640 1 -- 172.21.15.111:0/2580538621 learned_addr learned my addr 172.21.15.111:0/2580538621 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea85830640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea800b20a0 0x7fea800ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.467+0000 7fea86031640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800d1780 0x7fea800b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea8502f640 1 -- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea800b20a0 msgr2=0x7fea800ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea8502f640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea800b20a0 0x7fea800ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea8502f640 1 -- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800d1780 msgr2=0x7fea800b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea8502f640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800d1780 0x7fea800b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea8502f640 1 -- 172.21.15.111:0/2580538621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78009a70 con 0x7fea800d01e0 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea8502f640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea8015a580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c480 tx=0x7fea7000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:09.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70005ce0 con 0x7fea800d01e0 2023-12-04T05:41:09.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70005e40 con 0x7fea800d01e0 2023-12-04T05:41:09.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea800cb110 con 0x7fea800d01e0 2023-12-04T05:41:09.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700072f0 con 0x7fea800d01e0 2023-12-04T05:41:09.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.468+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8015aac0 con 0x7fea800d01e0 2023-12-04T05:41:09.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.469+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea800693d0 con 0x7fea800d01e0 2023-12-04T05:41:09.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.469+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fea70007510 con 0x7fea800d01e0 2023-12-04T05:41:09.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.470+0000 7fea767fc640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fea6406b2a0 0x7fea6406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:09.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.470+0000 7fea85830640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fea6406b2a0 0x7fea6406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:09.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.470+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea70088fc0 con 0x7fea800d01e0 2023-12-04T05:41:09.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.470+0000 7fea85830640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fea6406b2a0 0x7fea6406d760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea7c006fd0 tx=0x7fea7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:09.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.474+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea700bbe90 con 0x7fea800d01e0 2023-12-04T05:41:09.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.674+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fea800d40b0 con 0x7fea800d01e0 2023-12-04T05:41:09.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.674+0000 7fea767fc640 1 -- 172.21.15.111:0/2580538621 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7fea70017020 con 0x7fea800d01e0 2023-12-04T05:41:09.676 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-04T05:41:09.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fea6406b2a0 msgr2=0x7fea6406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:09.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fea6406b2a0 0x7fea6406d760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea7c006fd0 tx=0x7fea7c008040 comp rx=0 tx=0).stop 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 msgr2=0x7fea8015a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea8015a580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c480 tx=0x7fea7000c940 comp rx=0 tx=0).stop 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 shutdown_connections 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fea6406b2a0 0x7fea6406d760 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800d1780 0x7fea800b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea800d01e0 0x7fea8015a580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 --2- 172.21.15.111:0/2580538621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea800b20a0 0x7fea800ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:09.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 >> 172.21.15.111:0/2580538621 conn(0x7fea8009c4e0 msgr2=0x7fea800b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:09.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 shutdown_connections 2023-12-04T05:41:09.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:09.676+0000 7fea87a94640 1 -- 172.21.15.111:0/2580538621 wait complete. 2023-12-04T05:41:09.688 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-04T05:41:09.688 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-04T05:41:09.689 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:41:09.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.690 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.709 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-04T05:41:09.710 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42037sec preferred_lft 42037sec 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 1a:e1:a2:48:a2:71 brd ff:ff:ff:ff:ff:ff 2023-12-04T05:41:09.711 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::495:82ff:fe57:3769/64 scope link 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 1a:e1:a2:48:a2:71 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::18e1:a2ff:fe48:a271/64 scope link 2023-12-04T05:41:09.712 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.713 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-04T05:41:09.713 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether fe:d5:d3:8d:28:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:41:09.713 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::fcd5:d3ff:fe8d:28fa/64 scope link 2023-12-04T05:41:09.713 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-04T05:41:09.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.714 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-04T05:41:09.779 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-04T05:41:09.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.781 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-04T05:41:09.838 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-04T05:41:09.838 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-04T05:41:09.839 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-04T05:41:09.839 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-04T05:41:09.839 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-04T05:41:09.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.839 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-04T05:41:09.839 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-04T05:41:09.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-04T05:41:09.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-04T05:41:09.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-04T05:41:09.839 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-04T05:41:09.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.940 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-04T05:41:09.955 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-04T05:41:09.955 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-04T05:41:09.956 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:41:09.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:09.956 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-04T05:41:10.024 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-04T05:41:10.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:41:10.024 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-04T05:41:10.080 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-04T05:41:10.080 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-04T05:41:10.080 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-04T05:41:10.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:41:10.203 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:41:10.259 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:41:10.260 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:41:10.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:10.261 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:41:10.314 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-04T05:41:10.314 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-04T05:41:10.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-04T05:41:10.383 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2023-12-04T05:41:10.439 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2023-12-04T05:41:10.440 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:41:10.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:10.441 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-04T05:41:10.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-04T05:41:10.440+0000 7fea0703c480 -1 init, newargv = 0x556be6b4f440 newargc=15 2023-12-04T05:41:10.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[97146]: starting ceph client 2023-12-04T05:41:10.446 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[97146]: starting fuse 2023-12-04T05:41:10.457 INFO:teuthology.orchestra.run.smithi192.stdout:41 2023-12-04T05:41:10.457 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-04T05:41:10.457 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2023-12-04T05:41:10.457 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-04T05:41:10.457 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-04T05:41:10.457 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-04T05:41:10.457 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-04T05:41:10.457 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-04T05:41:10.458 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-04T05:41:10.459 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-04T05:41:10.459 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-04T05:41:10.459 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-04T05:41:10.459 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-04T05:41:10.459 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-04T05:41:10.576 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.97146.asok 2023-12-04T05:41:10.580 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-04T05:41:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-04T05:41:10.587 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.97146.asok 2023-12-04T05:41:10.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:10.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.97146.asok status 2023-12-04T05:41:10.722 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "47c1c547372ed5bff7984647dc200bac0e019caf", 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev)", 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "97146", 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5512, 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-04T05:41:10.723 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5512 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3021011628 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3021011628 2023-12-04T05:41:10.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5512 192.168.0.1:0/3021011628", 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/3021011628", 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 20, 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 45, 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-04T05:41:10.725 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-04T05:41:10.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:10.731 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-04T05:41:10.731 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-04T05:41:10.746 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-04T05:41:10.746 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-04T05:41:10.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:10.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-04T05:41:10.813 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-12-04T05:41:10.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.927+0000 7fda8298e640 1 Processor -- start 2023-12-04T05:41:10.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.927+0000 7fda8298e640 1 -- start start 2023-12-04T05:41:10.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.927+0000 7fda8298e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fda7c15d860 0x7fda7c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.927+0000 7fda8298e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c0d0010 0x7fda7c160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.927+0000 7fda8298e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c15ddc0 con 0x7fda7c15d480 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.927+0000 7fda8298e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c0d04c0 con 0x7fda7c15d860 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda8298e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15e0a0 con 0x7fda7c0d0010 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda80f2b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c0d0010 0x7fda7c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda7b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fda7c15d860 0x7fda7c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda7bfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fda7c15d480 0x7fda7c0d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:42772/0 (socket says 172.21.15.111:42772) 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda7bfff640 1 -- 172.21.15.111:0/209797072 learned_addr learned my addr 172.21.15.111:0/209797072 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda80f2b640 1 -- 172.21.15.111:0/209797072 >> v2:172.21.15.192:3300/0 conn(0x7fda7c15d860 msgr2=0x7fda7c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda80f2b640 1 --2- 172.21.15.111:0/209797072 >> v2:172.21.15.192:3300/0 conn(0x7fda7c15d860 0x7fda7c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:10.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda80f2b640 1 -- 172.21.15.111:0/209797072 >> v1:172.21.15.192:6789/0 conn(0x7fda7c15d480 legacy=0x7fda7c0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:10.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda80f2b640 1 -- 172.21.15.111:0/209797072 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c15e380 con 0x7fda7c0d0010 2023-12-04T05:41:10.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.928+0000 7fda7b7fe640 1 --2- 172.21.15.111:0/209797072 >> v2:172.21.15.192:3300/0 conn(0x7fda7c15d860 0x7fda7c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:41:10.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.929+0000 7fda80f2b640 1 --2- 172.21.15.111:0/209797072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c0d0010 0x7fda7c160270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda74009dc0 tx=0x7fda7402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=398fe69246965c31 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:10.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.929+0000 7fda7affd640 1 -- 172.21.15.111:0/209797072 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7402cc40 con 0x7fda7c0d0010 2023-12-04T05:41:10.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.929+0000 7fda7affd640 1 -- 172.21.15.111:0/209797072 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7402cda0 con 0x7fda7c0d0010 2023-12-04T05:41:10.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.929+0000 7fda7affd640 1 -- 172.21.15.111:0/209797072 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74034660 con 0x7fda7c0d0010 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 -- 172.21.15.111:0/209797072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c0d0010 msgr2=0x7fda7c160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 --2- 172.21.15.111:0/209797072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c0d0010 0x7fda7c160270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda74009dc0 tx=0x7fda7402be90 comp rx=0 tx=0).stop 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 -- 172.21.15.111:0/209797072 shutdown_connections 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 --2- 172.21.15.111:0/209797072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c0d0010 0x7fda7c160270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 --2- 172.21.15.111:0/209797072 >> v2:172.21.15.192:3300/0 conn(0x7fda7c15d860 0x7fda7c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 -- 172.21.15.111:0/209797072 >> 172.21.15.111:0/209797072 conn(0x7fda7c09c540 msgr2=0x7fda7c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 -- 172.21.15.111:0/209797072 shutdown_connections 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.930+0000 7fda8298e640 1 -- 172.21.15.111:0/209797072 wait complete. 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.931+0000 7fda8298e640 1 Processor -- start 2023-12-04T05:41:10.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.931+0000 7fda8298e640 1 -- start start 2023-12-04T05:41:10.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda8298e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda7c0d0010 0x7fda7c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:10.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda8298e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c15d480 0x7fda7c0b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:10.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda8298e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 0x7fda7c0b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:10.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda8298e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1499c0 con 0x7fda7c15d860 2023-12-04T05:41:10.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda8298e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0b1790 con 0x7fda7c0d0010 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda8298e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c149040 con 0x7fda7c15d480 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 0x7fda7c0b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 0x7fda7c0b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46326/0 (socket says 172.21.15.111:46326) 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 -- 172.21.15.111:0/3448128760 learned_addr learned my addr 172.21.15.111:0/3448128760 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda7b7fe640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c15d480 0x7fda7c0b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda7bfff640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda7c0d0010 0x7fda7c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 -- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c15d480 msgr2=0x7fda7c0b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c15d480 0x7fda7c0b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 -- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda7c0d0010 msgr2=0x7fda7c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda7c0d0010 0x7fda7c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 -- 172.21.15.111:0/3448128760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda70009400 con 0x7fda7c15d860 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.932+0000 7fda80f2b640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 0x7fda7c0b6580 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fda74009ee0 tx=0x7fda74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:10.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.933+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74040650 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.933+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda740407b0 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.933+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda74009a70 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.933+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74040b60 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.933+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c0cd100 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.934+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c067980 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.934+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fda74040cc0 con 0x7fda7c15d860 2023-12-04T05:41:10.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.934+0000 7fda78ff9640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fda6806b2a0 0x7fda6806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:10.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.935+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda740ac6d0 con 0x7fda7c15d860 2023-12-04T05:41:10.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.935+0000 7fda7bfff640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fda6806b2a0 0x7fda6806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:10.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.935+0000 7fda7bfff640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fda6806b2a0 0x7fda6806d760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fda6c005880 tx=0x7fda6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:10.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:10.939+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda74077680 con 0x7fda7c15d860 2023-12-04T05:41:11.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.140+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fda7c149b00 con 0x7fda7c15d860 2023-12-04T05:41:11.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.330+0000 7fda78ff9640 1 -- 172.21.15.111:0/3448128760 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v21) v1 ==== 105+0+0 (secure 0 0 0) 0x7fda7c149b00 con 0x7fda7c15d860 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fda6806b2a0 msgr2=0x7fda6806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fda6806b2a0 0x7fda6806d760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fda6c005880 tx=0x7fda6c009040 comp rx=0 tx=0).stop 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 msgr2=0x7fda7c0b6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 0x7fda7c0b6580 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fda74009ee0 tx=0x7fda74004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 shutdown_connections 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fda6806b2a0 0x7fda6806d760 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15d860 0x7fda7c0b6580 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda7c15d480 0x7fda7c0b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 --2- 172.21.15.111:0/3448128760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda7c0d0010 0x7fda7c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 >> 172.21.15.111:0/3448128760 conn(0x7fda7c09c540 msgr2=0x7fda7c15eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:11.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 shutdown_connections 2023-12-04T05:41:11.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.332+0000 7fda8298e640 1 -- 172.21.15.111:0/3448128760 wait complete. 2023-12-04T05:41:11.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:41:11.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 Processor -- start 2023-12-04T05:41:11.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 -- start start 2023-12-04T05:41:11.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4f415b480 0x7fd4f40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:11.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4f415f4e0 0x7fd4f40cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:11.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f40cbca0 con 0x7fd4f40cf9f0 2023-12-04T05:41:11.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f40cbf40 con 0x7fd4f415f4e0 2023-12-04T05:41:11.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.447+0000 7fd4f9e7c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f40cc220 con 0x7fd4f415b480 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4f415f4e0 0x7fd4f40cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4f415f4e0 0x7fd4f40cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46328/0 (socket says 172.21.15.111:46328) 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1579399226 learned_addr learned my addr 172.21.15.111:0/1579399226 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f37fe640 1 --2- 172.21.15.111:0/1579399226 >> v2:172.21.15.192:3300/0 conn(0x7fd4f415b480 0x7fd4f40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1579399226 >> v2:172.21.15.192:3300/0 conn(0x7fd4f415b480 msgr2=0x7fd4f40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 --2- 172.21.15.111:0/1579399226 >> v2:172.21.15.192:3300/0 conn(0x7fd4f415b480 0x7fd4f40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1579399226 >> v1:172.21.15.192:6789/0 conn(0x7fd4f40cf9f0 legacy=0x7fd4f415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.448+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1579399226 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40cc500 con 0x7fd4f415f4e0 2023-12-04T05:41:11.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f3fff640 1 --2- 172.21.15.111:0/1579399226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f40cca20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4071e00 tx=0x7fd4ec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8808cee5d641d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f2ffd640 1 -- 172.21.15.111:0/1579399226 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec02cb20 con 0x7fd4f415f4e0 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f2ffd640 1 -- 172.21.15.111:0/1579399226 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec02cc80 con 0x7fd4f415f4e0 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f2ffd640 1 -- 172.21.15.111:0/1579399226 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0343f0 con 0x7fd4f415f4e0 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1579399226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 msgr2=0x7fd4f40cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1579399226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f40cca20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4071e00 tx=0x7fd4ec02bf80 comp rx=0 tx=0).stop 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1579399226 shutdown_connections 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1579399226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f40cca20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1579399226 >> v2:172.21.15.192:3300/0 conn(0x7fd4f415b480 0x7fd4f40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1579399226 >> 172.21.15.111:0/1579399226 conn(0x7fd4f40b2a70 msgr2=0x7fd4f40b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:11.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1579399226 shutdown_connections 2023-12-04T05:41:11.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.449+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1579399226 wait complete. 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.450+0000 7fd4f9e7c640 1 Processor -- start 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 -- start start 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4f40cf9f0 0x7fd4f40c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f415b480 0x7fd4f40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f4159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40b4da0 con 0x7fd4f415f4e0 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40c4220 con 0x7fd4f415b480 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f9e7c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40c6a80 con 0x7fd4f40cf9f0 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f4159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f4159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46336/0 (socket says 172.21.15.111:46336) 2023-12-04T05:41:11.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1127655381 learned_addr learned my addr 172.21.15.111:0/1127655381 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4ebfff640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f415b480 0x7fd4f40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.451+0000 7fd4f37fe640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4f40cf9f0 0x7fd4f40c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4f40cf9f0 msgr2=0x7fd4f40c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f3fff640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4f40cf9f0 0x7fd4f40c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f415b480 msgr2=0x7fd4f40c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f3fff640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f415b480 0x7fd4f40c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f3fff640 1 -- 172.21.15.111:0/1127655381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec009b30 con 0x7fd4f415f4e0 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4ebfff640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f415b480 0x7fd4f40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f3fff640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f4159420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec036040 tx=0x7fd4ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec009510 con 0x7fd4f415f4e0 2023-12-04T05:41:11.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec02c920 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40c6d60 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec03e690 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.452+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f40b4a10 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.453+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f40b4f10 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.453+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd4ec03e7f0 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.454+0000 7fd4f0ff9640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd4d406b2a0 0x7fd4d406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.454+0000 7fd4f37fe640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd4d406b2a0 0x7fd4d406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.454+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4ec0ac620 con 0x7fd4f415f4e0 2023-12-04T05:41:11.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.454+0000 7fd4f37fe640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd4d406b2a0 0x7fd4d406d760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4f409f070 tx=0x7fd4e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:11.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.458+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4ec077520 con 0x7fd4f415f4e0 2023-12-04T05:41:11.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.658+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4f4067460 con 0x7fd4f415f4e0 2023-12-04T05:41:11.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.658+0000 7fd4f0ff9640 1 -- 172.21.15.111:0/1127655381 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fd4ec07ab40 con 0x7fd4f415f4e0 2023-12-04T05:41:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:11.660 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/956157809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":956157809},{"type":"v1","addr":"172.21.15.192:6835","nonce":956157809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:41:11.333367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5868,"mds_1":5865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5865":{"gid":5865,"name":"a","rank":1,"incarnation":22,"state":"up:creating","state_seq":1,"addr":"172.21.15.111:6836/619465105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":619465105},{"type":"v1","addr":"172.21.15.111:6836","nonce":619465105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:41:11.661 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 22 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd4d406b2a0 msgr2=0x7fd4d406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd4d406b2a0 0x7fd4d406d760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd4f409f070 tx=0x7fd4e4008040 comp rx=0 tx=0).stop 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 msgr2=0x7fd4f4159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f4159420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec036040 tx=0x7fd4ec004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 shutdown_connections 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd4d406b2a0 0x7fd4d406d760 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f415f4e0 0x7fd4f4159420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f415b480 0x7fd4f40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 --2- 172.21.15.111:0/1127655381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4f40cf9f0 0x7fd4f40c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:11.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 >> 172.21.15.111:0/1127655381 conn(0x7fd4f40b2a70 msgr2=0x7fd4f406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:11.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 shutdown_connections 2023-12-04T05:41:11.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:11.660+0000 7fd4f9e7c640 1 -- 172.21.15.111:0/1127655381 wait complete. 2023-12-04T05:41:12.075 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-04T05:41:02.644246+0000', 'modified': '2023-12-04T05:41:11.333367+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5868, 'mds_1': 5865}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5865': {'gid': 5865, 'name': 'a', 'rank': 1, 'incarnation': 22, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.111:6836/619465105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 619465105}, {'type': 'v1', 'addr': '172.21.15.111:6836', 'nonce': 619465105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5868': {'gid': 5868, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6837/2442390644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6835', 'nonce': 2442390644}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 2442390644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-04T05:41:12.075 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5865:up:creating 2023-12-04T05:41:13.076 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:41:13.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.185+0000 7f481014c640 1 Processor -- start 2023-12-04T05:41:13.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.186+0000 7f481014c640 1 -- start start 2023-12-04T05:41:13.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.186+0000 7f481014c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48080d1780 0x7f48080d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.186+0000 7f481014c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48080ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.186+0000 7f481014c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48080cdcb0 con 0x7f48080b20a0 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.186+0000 7f481014c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080d39e0 con 0x7f48080d1780 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.186+0000 7f481014c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080d3cc0 con 0x7f48080b2ad0 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48080d1780 0x7f48080d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48080d1780 0x7f48080d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46338/0 (socket says 172.21.15.111:46338) 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 -- 172.21.15.111:0/235726223 learned_addr learned my addr 172.21.15.111:0/235726223 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480e6e9640 1 --2- 172.21.15.111:0/235726223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48080ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 -- 172.21.15.111:0/235726223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 msgr2=0x7f48080ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 --2- 172.21.15.111:0/235726223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48080ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 -- 172.21.15.111:0/235726223 >> v1:172.21.15.192:6789/0 conn(0x7f48080b20a0 legacy=0x7f48080d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.187+0000 7f480dee8640 1 -- 172.21.15.111:0/235726223 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080cdf50 con 0x7f48080d1780 2023-12-04T05:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f480dee8640 1 --2- 172.21.15.111:0/235726223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080d1780 0x7f48080d01e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4800009e50 tx=0x7f480002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3f5cae8cc0e333 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.190 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f480cee6640 1 -- 172.21.15.111:0/235726223 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480002ca70 con 0x7f48080d1780 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f480cee6640 1 -- 172.21.15.111:0/235726223 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480002cbd0 con 0x7f48080d1780 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f480cee6640 1 -- 172.21.15.111:0/235726223 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000343f0 con 0x7f48080d1780 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f481014c640 1 -- 172.21.15.111:0/235726223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080d1780 msgr2=0x7f48080d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f481014c640 1 --2- 172.21.15.111:0/235726223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080d1780 0x7f48080d01e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4800009e50 tx=0x7f480002bed0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f481014c640 1 -- 172.21.15.111:0/235726223 shutdown_connections 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f481014c640 1 --2- 172.21.15.111:0/235726223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48080ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f481014c640 1 --2- 172.21.15.111:0/235726223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080d1780 0x7f48080d01e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.188+0000 7f481014c640 1 -- 172.21.15.111:0/235726223 >> 172.21.15.111:0/235726223 conn(0x7f480809c4e0 msgr2=0x7f48080d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.189+0000 7f481014c640 1 -- 172.21.15.111:0/235726223 shutdown_connections 2023-12-04T05:41:13.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.189+0000 7f481014c640 1 -- 172.21.15.111:0/235726223 wait complete. 2023-12-04T05:41:13.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 Processor -- start 2023-12-04T05:41:13.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 -- start start 2023-12-04T05:41:13.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 0x7f480815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48081f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48080d1780 0x7f48081f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081f2e20 con 0x7f48080b20a0 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081f74a0 con 0x7f48080d1780 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.190+0000 7f481014c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48081f6ee0 con 0x7f48080b2ad0 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 0x7f480815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 0x7f480815c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46352/0 (socket says 172.21.15.111:46352) 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 -- 172.21.15.111:0/2364935145 learned_addr learned my addr 172.21.15.111:0/2364935145 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480d6e7640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48081f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480e6e9640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48080d1780 0x7f48081f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 -- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 msgr2=0x7f48081f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48081f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 -- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48080d1780 msgr2=0x7f48081f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48080d1780 0x7f48081f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 -- 172.21.15.111:0/2364935145 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009b30 con 0x7f48080b20a0 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f480dee8640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 0x7f480815c280 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4800000f80 tx=0x7f4800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800009510 con 0x7f48080b20a0 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480003e450 con 0x7f48080b20a0 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480003e800 con 0x7f48080b20a0 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48081f71c0 con 0x7f48080b20a0 2023-12-04T05:41:13.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.191+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48081f3390 con 0x7f48080b20a0 2023-12-04T05:41:13.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.192+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080693d0 con 0x7f48080b20a0 2023-12-04T05:41:13.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.193+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f480003e960 con 0x7f48080b20a0 2023-12-04T05:41:13.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.193+0000 7f47fe7fc640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47ec06b2a0 0x7f47ec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.193+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48000ac8b0 con 0x7f48080b20a0 2023-12-04T05:41:13.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.193+0000 7f480d6e7640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47ec06b2a0 0x7f47ec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.197 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.193+0000 7f480d6e7640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47ec06b2a0 0x7f47ec06d760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48080b4830 tx=0x7f47f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.197+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48000778f0 con 0x7f48080b20a0 2023-12-04T05:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.404+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48080a0f00 con 0x7f48080b20a0 2023-12-04T05:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.404+0000 7f47fe7fc640 1 -- 172.21.15.111:0/2364935145 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f480007af10 con 0x7f48080b20a0 2023-12-04T05:41:13.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:41:13.406 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/956157809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":956157809},{"type":"v1","addr":"172.21.15.192:6835","nonce":956157809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:41:11.345378+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5868,"mds_1":5865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5865":{"gid":5865,"name":"a","rank":1,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6836/619465105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":619465105},{"type":"v1","addr":"172.21.15.111:6836","nonce":619465105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 23 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47ec06b2a0 msgr2=0x7f47ec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47ec06b2a0 0x7f47ec06d760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48080b4830 tx=0x7f47f8009040 comp rx=0 tx=0).stop 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 msgr2=0x7f480815c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 0x7f480815c280 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4800000f80 tx=0x7f4800004a40 comp rx=0 tx=0).stop 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 shutdown_connections 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f47ec06b2a0 0x7f47ec06d760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48080d1780 0x7f48081f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48080b2ad0 0x7f48081f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 --2- 172.21.15.111:0/2364935145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48080b20a0 0x7f480815c280 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 >> 172.21.15.111:0/2364935145 conn(0x7f480809c4e0 msgr2=0x7f480806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 shutdown_connections 2023-12-04T05:41:13.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.406+0000 7f481014c640 1 -- 172.21.15.111:0/2364935145 wait complete. 2023-12-04T05:41:13.574 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:41:13.573+0000 7f344645d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-04T05:41:13.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-04T05:41:02.644246+0000', 'modified': '2023-12-04T05:41:11.345378+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5868, 'mds_1': 5865}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5865': {'gid': 5865, 'name': 'a', 'rank': 1, 'incarnation': 22, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6836/619465105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 619465105}, {'type': 'v1', 'addr': '172.21.15.111:6836', 'nonce': 619465105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_5868': {'gid': 5868, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6837/2442390644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6835', 'nonce': 2442390644}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 2442390644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-04T05:41:13.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-12-04T05:41:13.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-04T05:41:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.922+0000 7f10f6ecf640 1 Processor -- start 2023-12-04T05:41:13.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.922+0000 7f10f6ecf640 1 -- start start 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10f6ecf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10f00ceed0 0x7f10f00cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10f6ecf640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00ce110 con 0x7f10f00d1e40 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10f6ecf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00ce3f0 con 0x7f10f015d510 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10f6ecf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00ceab0 con 0x7f10f00ceed0 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10f546c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f10f00d1e40 0x7f10f00d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56974/0 (socket says 172.21.15.111:56974) 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10f546c640 1 -- 172.21.15.111:0/2538776318 learned_addr learned my addr 172.21.15.111:0/2538776318 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10effff640 1 --2- 172.21.15.111:0/2538776318 >> v2:172.21.15.192:3300/0 conn(0x7f10f00ceed0 0x7f10f00cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218799368 0 0) 0x7f10f00ce110 con 0x7f10f00d1e40 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e0003520 con 0x7f10f00d1e40 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226564355 0 0) 0x7f10e0003520 con 0x7f10f00d1e40 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 >> v2:172.21.15.192:3300/0 conn(0x7f10f00ceed0 msgr2=0x7f10f00cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 --2- 172.21.15.111:0/2538776318 >> v2:172.21.15.192:3300/0 conn(0x7f10f00ceed0 0x7f10f00cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 >> v1:172.21.15.192:6789/0 conn(0x7f10f015d510 legacy=0x7f10f00d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00ce6d0 con 0x7f10f00d1e40 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f10e80019f0 con 0x7f10f00d1e40 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 >> v1:172.21.15.111:6789/0 conn(0x7f10f00d1e40 legacy=0x7f10f00d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e0003970 0x7f10e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e0005180 0x7f10e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00ceab0 con 0x7f10e00047c0 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000c020 con 0x7f10e0005180 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.923+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000c1e0 con 0x7f10e0003970 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10effff640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e0003970 0x7f10e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10f546c640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e0005180 0x7f10e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10effff640 1 -- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e0003970 msgr2=0x7f10e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10effff640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e0003970 0x7f10e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10effff640 1 -- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e0005180 msgr2=0x7f10e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10effff640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e0005180 0x7f10e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10effff640 1 -- 172.21.15.111:0/2538776318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0003520 con 0x7f10e00047c0 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e0003970 0x7f10e0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10f546c640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e0005180 0x7f10e000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.924+0000 7f10f546c640 1 -- 172.21.15.111:0/2538776318 reap_dead start 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10effff640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10e0004c40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009ac0 tx=0x7f10e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c755cdf5073544c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc02c6b0 con 0x7f10e00047c0 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc02c810 con 0x7f10e00047c0 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10ef7fe640 1 -- 172.21.15.111:0/2538776318 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc02cba0 con 0x7f10e00047c0 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 -- 172.21.15.111:0/2538776318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 msgr2=0x7f10e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10e0004c40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009ac0 tx=0x7f10e00057f0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 -- 172.21.15.111:0/2538776318 shutdown_connections 2023-12-04T05:41:13.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/2538776318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10e0004c40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 -- 172.21.15.111:0/2538776318 >> 172.21.15.111:0/2538776318 conn(0x7f10f009c460 msgr2=0x7f10f00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 -- 172.21.15.111:0/2538776318 shutdown_connections 2023-12-04T05:41:13.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.925+0000 7f10f6ecf640 1 -- 172.21.15.111:0/2538776318 wait complete. 2023-12-04T05:41:13.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.926+0000 7f10f6ecf640 1 Processor -- start 2023-12-04T05:41:13.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.926+0000 7f10f6ecf640 1 -- start start 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f6ecf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10f009dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f6ecf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f009ced0 0x7f10f00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f6ecf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10f00cc330 0x7f10f00c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f6ecf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00cc7e0 con 0x7f10e00047c0 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f6ecf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f009d620 con 0x7f10f009ced0 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f6ecf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f009d900 con 0x7f10f00cc330 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f4c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10f009dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f4c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10f009dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46372/0 (socket says 172.21.15.111:46372) 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.927+0000 7f10f4c6b640 1 -- 172.21.15.111:0/3171176592 learned_addr learned my addr 172.21.15.111:0/3171176592 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10effff640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f009ced0 0x7f10f00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f546c640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10f00cc330 0x7f10f00c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f4c6b640 1 -- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10f00cc330 msgr2=0x7f10f00c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10f00cc330 0x7f10f00c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f4c6b640 1 -- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f009ced0 msgr2=0x7f10f00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f009ced0 0x7f10f00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f4c6b640 1 -- 172.21.15.111:0/3171176592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f009dbe0 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10effff640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f009ced0 0x7f10f00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10f009dec0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e480 tx=0x7f10e400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10ed7fa640 1 -- 172.21.15.111:0/3171176592 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4017070 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10ed7fa640 1 -- 172.21.15.111:0/3171176592 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e40052e0 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00c6e40 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10ed7fa640 1 -- 172.21.15.111:0/3171176592 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4005490 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.928+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f00c78a0 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.930+0000 7f10ed7fa640 1 -- 172.21.15.111:0/3171176592 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f10e4007020 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.930+0000 7f10ed7fa640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10d406b2a0 0x7f10d406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.930+0000 7f10ed7fa640 1 -- 172.21.15.111:0/3171176592 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e4089160 con 0x7f10e00047c0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.930+0000 7f10effff640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10d406b2a0 0x7f10d406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.930+0000 7f10effff640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10d406b2a0 0x7f10d406d760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f10dc036000 tx=0x7f10dc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.931+0000 7f10f9a67740 1 Processor -- start 2023-12-04T05:41:13.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.931+0000 7f10f9a67740 1 -- start start 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0d75f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da3380 0x561ec0da5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 0x561ec0da6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f9a67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0d36d10 con 0x561ec0d76480 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0ce8b90 con 0x561ec0da3380 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec0ce6280 con 0x561ec0da2fa0 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 0x561ec0da6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 0x561ec0da6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46378/0 (socket says 172.21.15.111:46378) 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 -- 172.21.15.111:0/2654374236 learned_addr learned my addr 172.21.15.111:0/2654374236 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10effff640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da3380 0x561ec0da5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0d75f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 -- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 msgr2=0x561ec0d75f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0d75f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 -- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da3380 msgr2=0x561ec0da5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da3380 0x561ec0da5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 -- 172.21.15.111:0/2654374236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec0ca7500 con 0x561ec0d76480 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10effff640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da3380 0x561ec0da5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0d75f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10f546c640 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 0x561ec0da6270 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f10f00cf2b0 tx=0x7f10e8005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c84617f3c292ebb server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10da7fc640 1 -- 172.21.15.111:0/2654374236 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800d070 con 0x561ec0d76480 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10da7fc640 1 -- 172.21.15.111:0/2654374236 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8007840 con 0x561ec0d76480 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.932+0000 7f10da7fc640 1 -- 172.21.15.111:0/2654374236 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8014430 con 0x561ec0d76480 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 -- 172.21.15.111:0/2654374236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 msgr2=0x561ec0da6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 0x561ec0da6270 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f10f00cf2b0 tx=0x7f10e8005f40 comp rx=0 tx=0).stop 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 -- 172.21.15.111:0/2654374236 shutdown_connections 2023-12-04T05:41:13.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0d76480 0x561ec0da6270 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da3380 0x561ec0da5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 --2- 172.21.15.111:0/2654374236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0d75f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 -- 172.21.15.111:0/2654374236 >> 172.21.15.111:0/2654374236 conn(0x561ec0d781b0 msgr2=0x561ec0dc0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 -- 172.21.15.111:0/2654374236 shutdown_connections 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.933+0000 7f10f9a67740 1 -- 172.21.15.111:0/2654374236 wait complete. 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.934+0000 7f10f9a67740 1 -- start start 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.934+0000 7f10f9a67740 1 Processor -- start 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da2fa0 0x561ec0d90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0da9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0dbb360 0x561ec0d93490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0ce1b70 con 0x561ec0da3380 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0c9be00 con 0x561ec0da2fa0 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec0c9baa0 con 0x561ec0dbb360 2023-12-04T05:41:13.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 10 client.0 _ll_register_callbacks cb 0x561ec0ca02d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0da9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0da9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46382/0 (socket says 172.21.15.111:46382) 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 -- 172.21.15.111:0/91024601 learned_addr learned my addr 172.21.15.111:0/91024601 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da2fa0 0x561ec0d90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f546c640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0dbb360 0x561ec0d93490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 -- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0dbb360 msgr2=0x561ec0d93490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0dbb360 0x561ec0d93490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 -- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da2fa0 msgr2=0x561ec0d90cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da2fa0 0x561ec0d90cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 -- 172.21.15.111:0/91024601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f00ce6d0 con 0x561ec0da3380 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10effff640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0da9ba0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009be0 tx=0x7f10dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f14117580d4468 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10cf7fe640 1 -- 172.21.15.111:0/91024601 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc003b60 con 0x561ec0da3380 2023-12-04T05:41:13.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10cf7fe640 1 -- 172.21.15.111:0/91024601 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10dc003cc0 con 0x561ec0da3380 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ec0cdab50 con 0x561ec0da3380 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10cf7fe640 1 -- 172.21.15.111:0/91024601 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10dc003e80 con 0x561ec0da3380 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 10 client.6063 fetch_fsmap learned FSMap version 23 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.935+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561ec0ce01f0 con 0x561ec0da3380 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10cf7fe640 1 -- 172.21.15.111:0/91024601 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ec0cdab50 con 0x561ec0da3380 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10cf7fe640 1 -- 172.21.15.111:0/91024601 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f10dc048480 con 0x561ec0da3380 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f9a67740 10 client.6063 fetch_fsmap finished waiting for FSMap version 23 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f9a67740 10 client.6063 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f9a67740 20 client.6063 populate_metadata read hostname 'smithi111' 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f9a67740 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0db8e30 0x561ec0db9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f9a67740 4 client.6063 mds_command: new command op to 5865 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 --> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ec0cda870 con 0x561ec0db8e30 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0db8e30 0x561ec0db9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0db8e30 0x561ec0db9210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6688a84e91a2839c server_cookie=11ddd3422d3a2b0e in_seq=0 out_seq=0 2023-12-04T05:41:13.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.936+0000 7f10cf7fe640 10 client.6063 ms_handle_connect on v2:172.21.15.111:6834/619465105 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10cf7fe640 1 -- 172.21.15.111:0/91024601 <== mds.1 v2:172.21.15.111:6834/619465105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ec0cda870 con 0x561ec0db8e30 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10cf7fe640 10 client.6063 handle_command_reply: tid=0 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 client.6063 shutdown 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 msgr2=0x561ec0da9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0da9ba0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009be0 tx=0x7f10dc0046c0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 shutdown_connections 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0db8e30 0x561ec0db9210 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0dbb360 0x561ec0d93490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0da9ba0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 --2- 172.21.15.111:0/91024601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0da2fa0 0x561ec0d90cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10cf7fe640 0 client.6063 ms_handle_reset on v2:172.21.15.111:6834/619465105 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 >> 172.21.15.111:0/91024601 conn(0x561ec0d76480 msgr2=0x561ec0dc4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 shutdown_connections 2023-12-04T05:41:13.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 1 -- 172.21.15.111:0/91024601 wait complete. 2023-12-04T05:41:13.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.937+0000 7f10f9a67740 20 client.6063 trim_cache size 0 max 16384 2023-12-04T05:41:13.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.952+0000 7f10f9a67740 1 Processor -- start 2023-12-04T05:41:13.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.952+0000 7f10f9a67740 1 -- start start 2023-12-04T05:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0da6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0d79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d7a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10f9a67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0c61270 con 0x561ec0da3380 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0d97720 con 0x561ec0dbb360 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec0d751d0 con 0x561ec0da2fa0 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10effff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0d79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10effff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0d79b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46394/0 (socket says 172.21.15.111:46394) 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10effff640 1 -- 172.21.15.111:0/3076586900 learned_addr learned my addr 172.21.15.111:0/3076586900 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10effff640 1 -- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 msgr2=0x561ec0da6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.953+0000 7f10effff640 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0da6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f546c640 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d7a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10effff640 1 -- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 msgr2=0x561ec0d7a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10effff640 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d7a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10effff640 1 -- 172.21.15.111:0/3076586900 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec0dc1da0 con 0x561ec0da3380 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f546c640 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d7a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:13.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10effff640 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0d79b40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f10dc054040 tx=0x7f10dc047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec65b3446025a3ec server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10daffd640 1 -- 172.21.15.111:0/3076586900 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00e030 con 0x561ec0da3380 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10daffd640 1 -- 172.21.15.111:0/3076586900 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc048990 con 0x561ec0da3380 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10daffd640 1 -- 172.21.15.111:0/3076586900 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc048d20 con 0x561ec0da3380 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 -- 172.21.15.111:0/3076586900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 msgr2=0x561ec0d79b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0d79b40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f10dc054040 tx=0x7f10dc047000 comp rx=0 tx=0).stop 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 -- 172.21.15.111:0/3076586900 shutdown_connections 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d7a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da3380 0x561ec0d79b40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 --2- 172.21.15.111:0/3076586900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da2fa0 0x561ec0da6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 -- 172.21.15.111:0/3076586900 >> 172.21.15.111:0/3076586900 conn(0x561ec0d76480 msgr2=0x561ec0dbbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 -- 172.21.15.111:0/3076586900 shutdown_connections 2023-12-04T05:41:13.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.954+0000 7f10f9a67740 1 -- 172.21.15.111:0/3076586900 wait complete. 2023-12-04T05:41:13.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.955+0000 7f10f9a67740 1 -- start start 2023-12-04T05:41:13.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.955+0000 7f10f9a67740 1 Processor -- start 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 0x561ec0dc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da3380 0x561ec0dc2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d8f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0dc4d20 con 0x561ec0da2fa0 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec0dc2a90 con 0x561ec0dbb360 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ec0dd2960 con 0x561ec0da3380 2023-12-04T05:41:13.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f9a67740 10 client.0 _ll_register_callbacks cb 0x7f10e0005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 0x561ec0dc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 0x561ec0dc47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46398/0 (socket says 172.21.15.111:46398) 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 -- 172.21.15.111:0/551997709 learned_addr learned my addr 172.21.15.111:0/551997709 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10effff640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da3380 0x561ec0dc2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f546c640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d8f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 -- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da3380 msgr2=0x561ec0dc2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da3380 0x561ec0dc2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 -- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 msgr2=0x561ec0d8f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d8f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 -- 172.21.15.111:0/551997709 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ec0ce01f0 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10f4c6b640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 0x561ec0dc47e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f10e4057a20 tx=0x7f10e4057d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159222f284208960 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.956+0000 7f10cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10cf7fe640 1 -- 172.21.15.111:0/551997709 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4060020 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10cf7fe640 1 -- 172.21.15.111:0/551997709 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e4054430 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10e8007840 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10cf7fe640 1 -- 172.21.15.111:0/551997709 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e40545f0 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 10 client.6069 fetch_fsmap learned FSMap version 23 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x561ec0c7dd00 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10cf7fe640 1 -- 172.21.15.111:0/551997709 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8007840 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10cf7fe640 1 -- 172.21.15.111:0/551997709 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f10e4087270 con 0x561ec0da2fa0 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 10 client.6069 fetch_fsmap finished waiting for FSMap version 23 2023-12-04T05:41:13.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 10 client.6069 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 20 client.6069 populate_metadata read hostname 'smithi111' 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0e2ac70 0x561ec0d8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 4 client.6069 mds_command: new command op to 5865 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 --> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561ec09e26c0 con 0x561ec0e2ac70 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10effff640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0e2ac70 0x561ec0d8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10cf7fe640 10 client.6069 ms_handle_connect on v2:172.21.15.111:6834/619465105 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.957+0000 7f10effff640 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0e2ac70 0x561ec0d8fd10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35eacaaabb94d109 server_cookie=7102431152fb13b9 in_seq=0 out_seq=0 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10cf7fe640 1 -- 172.21.15.111:0/551997709 <== mds.1 v2:172.21.15.111:6834/619465105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561ec09e26c0 con 0x561ec0e2ac70 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10cf7fe640 10 client.6069 handle_command_reply: tid=0 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 client.6069 shutdown 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 msgr2=0x561ec0dc47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 0x561ec0dc47e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f10e4057a20 tx=0x7f10e4057d20 comp rx=0 tx=0).stop 2023-12-04T05:41:13.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 shutdown_connections 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:6834/619465105,v1:172.21.15.111:6836/619465105] conn(0x561ec0e2ac70 0x561ec0d8fd10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ec0dbb360 0x561ec0d8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x561ec0da3380 0x561ec0dc2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 --2- 172.21.15.111:0/551997709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ec0da2fa0 0x561ec0dc47e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10cf7fe640 0 client.6069 ms_handle_reset on v2:172.21.15.111:6834/619465105 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 >> 172.21.15.111:0/551997709 conn(0x561ec0d76480 msgr2=0x561ec0d79b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 shutdown_connections 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 1 -- 172.21.15.111:0/551997709 wait complete. 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f9a67740 20 client.6069 trim_cache size 0 max 16384 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:41:13.966 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "9686316e-a01b-4981-b0b7-e21bdb185d5e", 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 1, 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5865, 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 2.6236851200000002, 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 45, 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 45, 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 15.706458534999999 2023-12-04T05:41:13.967 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10d406b2a0 msgr2=0x7f10d406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10d406b2a0 0x7f10d406d760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f10dc036000 tx=0x7f10dc030000 comp rx=0 tx=0).stop 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 msgr2=0x7f10f009dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.958+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10f009dec0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e480 tx=0x7f10e400e940 comp rx=0 tx=0).stop 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 shutdown_connections 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f10d406b2a0 0x7f10d406d760 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10f00cc330 0x7f10f00c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f009ced0 0x7f10f00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 --2- 172.21.15.111:0/3171176592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e00047c0 0x7f10f009dec0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 >> 172.21.15.111:0/3171176592 conn(0x7f10f009c460 msgr2=0x7f10f015ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 shutdown_connections 2023-12-04T05:41:13.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:13.959+0000 7f10f6ecf640 1 -- 172.21.15.111:0/3171176592 wait complete. 2023-12-04T05:41:13.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-04T05:41:14.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.070+0000 7fe804989640 1 Processor -- start 2023-12-04T05:41:14.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.070+0000 7fe804989640 1 -- start start 2023-12-04T05:41:14.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe804989640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 0x7fe8000d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe804989640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe8000b20a0 0x7fe8000d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe804989640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8000cdcb0 con 0x7fe8000b2ad0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe804989640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8000d20a0 con 0x7fe8000b20a0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe804989640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000d39e0 con 0x7fe8000d1780 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe7fed9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe8000b2ad0 0x7fe8000ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56976/0 (socket says 172.21.15.111:56976) 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.071+0000 7fe7fed9d640 1 -- 172.21.15.111:0/4261551209 learned_addr learned my addr 172.21.15.111:0/4261551209 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 0x7fe8000d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/4261551209 >> v2:172.21.15.192:3300/0 conn(0x7fe8000b20a0 0x7fe8000d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243840417 0 0) 0x7fe8000cdcb0 con 0x7fe8000b2ad0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e8003520 con 0x7fe8000b2ad0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094594249 0 0) 0x7fe7e8003520 con 0x7fe8000b2ad0 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 msgr2=0x7fe8000d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 0x7fe8000d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 >> v2:172.21.15.192:3300/0 conn(0x7fe8000b20a0 msgr2=0x7fe8000d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 --2- 172.21.15.111:0/4261551209 >> v2:172.21.15.192:3300/0 conn(0x7fe8000b20a0 0x7fe8000d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8000d3cc0 con 0x7fe8000b2ad0 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fe7f8001670 con 0x7fe8000b2ad0 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/4261551209 >> v2:172.21.15.192:3300/0 conn(0x7fe8000b20a0 0x7fe8000d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 >> v1:172.21.15.111:6789/0 conn(0x7fe8000b2ad0 legacy=0x7fe8000ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 0x7fe7e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000d20a0 con 0x7fe7e8004d00 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8000d39e0 con 0x7fe7e8003970 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e800b980 con 0x7fe8000d1780 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 0x7fe7e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fed9d640 1 -- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 msgr2=0x7fe7e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000d1780 0x7fe7e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fed9d640 1 -- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e8003970 msgr2=0x7fe7e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.072+0000 7fe7fed9d640 1 -- 172.21.15.111:0/4261551209 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8003520 con 0x7fe7e8004d00 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.073+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c240 tx=0x7fe7e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abf9cfc28b9a81d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.073+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f802cb00 con 0x7fe7e8004d00 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.073+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f802cc60 con 0x7fe7e8004d00 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.073+0000 7fe7fd59a640 1 -- 172.21.15.111:0/4261551209 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8005b10 con 0x7fe7e8004d00 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe804989640 1 -- 172.21.15.111:0/4261551209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 msgr2=0x7fe7e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe804989640 1 --2- 172.21.15.111:0/4261551209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c240 tx=0x7fe7e80057f0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe7fed9d640 1 -- 172.21.15.111:0/4261551209 reap_dead start 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe804989640 1 -- 172.21.15.111:0/4261551209 shutdown_connections 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe804989640 1 -- 172.21.15.111:0/4261551209 >> 172.21.15.111:0/4261551209 conn(0x7fe80009c4e0 msgr2=0x7fe8000d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:14.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe804989640 1 -- 172.21.15.111:0/4261551209 shutdown_connections 2023-12-04T05:41:14.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.074+0000 7fe804989640 1 -- 172.21.15.111:0/4261551209 wait complete. 2023-12-04T05:41:14.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.075+0000 7fe804989640 1 Processor -- start 2023-12-04T05:41:14.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.075+0000 7fe804989640 1 -- start start 2023-12-04T05:41:14.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe804989640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe800152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe804989640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe80015dbb0 0x7fe8000cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe804989640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8001516d0 0x7fe800151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe804989640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015e390 con 0x7fe7e8004d00 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe804989640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015e670 con 0x7fe80015dbb0 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe804989640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015e950 con 0x7fe8001516d0 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe800152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe800152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46422/0 (socket says 172.21.15.111:46422) 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 -- 172.21.15.111:0/33113854 learned_addr learned my addr 172.21.15.111:0/33113854 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe80015dbb0 0x7fe8000cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8001516d0 0x7fe800151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 -- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8001516d0 msgr2=0x7fe800151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8001516d0 0x7fe800151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 -- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe80015dbb0 msgr2=0x7fe8000cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe80015dbb0 0x7fe8000cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 -- 172.21.15.111:0/33113854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80015ec30 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8001516d0 0x7fe800151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.076+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe800152760 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0011e30 tx=0x7fe7f000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.077+0000 7fe7eeffd640 1 -- 172.21.15.111:0/33113854 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000e040 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.077+0000 7fe7eeffd640 1 -- 172.21.15.111:0/33113854 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f000f040 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.077+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80015ef10 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.077+0000 7fe7eeffd640 1 -- 172.21.15.111:0/33113854 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0009c00 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.077+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80015e030 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.078+0000 7fe7eeffd640 1 -- 172.21.15.111:0/33113854 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe7f0015850 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.079+0000 7fe7eeffd640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7e006b2a0 0x7fe7e006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.079+0000 7fe7eeffd640 1 -- 172.21.15.111:0/33113854 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f008aa00 con 0x7fe7e8004d00 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.079+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7e006b2a0 0x7fe7e006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.079+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7e006b2a0 0x7fe7e006d760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe8000668f0 tx=0x7fe7f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.080+0000 7fe8075f9740 1 Processor -- start 2023-12-04T05:41:14.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.080+0000 7fe8075f9740 1 -- start start 2023-12-04T05:41:14.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739925fa0 0x5567398f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 0x556739928d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5567398f9480 0x556739929270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe8075f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567398b9d10 con 0x556739926380 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673986bb90 con 0x556739925fa0 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x556739869280 con 0x5567398f9480 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 0x556739928d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 0x556739928d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46424/0 (socket says 172.21.15.111:46424) 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 -- 172.21.15.111:0/2030011876 learned_addr learned my addr 172.21.15.111:0/2030011876 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739925fa0 0x5567398f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5567398f9480 0x556739929270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 -- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5567398f9480 msgr2=0x556739929270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5567398f9480 0x556739929270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 -- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739925fa0 msgr2=0x5567398f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739925fa0 0x5567398f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 -- 172.21.15.111:0/2030011876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55673982a500 con 0x556739926380 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739925fa0 0x5567398f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 0x556739928d30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4016460 tx=0x7fe7f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239230f71751342f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7dffff640 1 -- 172.21.15.111:0/2030011876 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4023660 con 0x556739926380 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7dffff640 1 -- 172.21.15.111:0/2030011876 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f40237c0 con 0x556739926380 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.081+0000 7fe7dffff640 1 -- 172.21.15.111:0/2030011876 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4023b50 con 0x556739926380 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 -- 172.21.15.111:0/2030011876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 msgr2=0x556739928d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 0x556739928d30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4016460 tx=0x7fe7f4017000 comp rx=0 tx=0).stop 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 -- 172.21.15.111:0/2030011876 shutdown_connections 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5567398f9480 0x556739929270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739926380 0x556739928d30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2030011876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739925fa0 0x5567398f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 -- 172.21.15.111:0/2030011876 >> 172.21.15.111:0/2030011876 conn(0x5567398fb1b0 msgr2=0x556739943d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 -- 172.21.15.111:0/2030011876 shutdown_connections 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.082+0000 7fe8075f9740 1 -- 172.21.15.111:0/2030011876 wait complete. 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.083+0000 7fe8075f9740 1 -- start start 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.083+0000 7fe8075f9740 1 Processor -- start 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x55673993ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739913c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739947240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739864b70 con 0x556739925fa0 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55673981ee00 con 0x55673993e360 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673981eaa0 con 0x556739926380 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 10 client.0 _ll_register_callbacks cb 0x5567398232d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x55673993ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x55673993ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46430/0 (socket says 172.21.15.111:46430) 2023-12-04T05:41:14.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 -- 172.21.15.111:0/243767736 learned_addr learned my addr 172.21.15.111:0/243767736 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739913c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739947240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 -- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 msgr2=0x556739913c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739913c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 -- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 msgr2=0x556739947240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739947240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 -- 172.21.15.111:0/243767736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f4009d10 con 0x556739925fa0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739913c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x55673993ae90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00594c0 tx=0x7fe7f005d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9641d00d1cbb73d server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/243767736 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0057990 con 0x556739925fa0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/243767736 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f005d160 con 0x556739925fa0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55673985db50 con 0x556739925fa0 2023-12-04T05:41:14.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/243767736 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f005d320 con 0x556739925fa0 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 10 client.6081 fetch_fsmap learned FSMap version 23 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.084+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5567398631f0 con 0x556739925fa0 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/243767736 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55673985db50 con 0x556739925fa0 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/243767736 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe7f001b020 con 0x556739925fa0 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe8075f9740 10 client.6081 fetch_fsmap finished waiting for FSMap version 23 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe8075f9740 10 client.6081 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe8075f9740 20 client.6081 populate_metadata read hostname 'smithi111' 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe8075f9740 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55673992ad70 0x55673992b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe8075f9740 4 client.6081 mds_command: new command op to 5868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 --> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55673985d870 con 0x55673992ad70 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55673992ad70 0x55673992b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe7dd7fa640 10 client.6081 ms_handle_connect on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.085+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55673992ad70 0x55673992b150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfad1f92ff5260a server_cookie=5672c2a836c9dd7b in_seq=0 out_seq=0 2023-12-04T05:41:14.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/243767736 <== mds.0 v2:172.21.15.111:6835/2442390644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55673985d870 con 0x55673992ad70 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe7dd7fa640 10 client.6081 handle_command_reply: tid=0 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 client.6081 shutdown 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 msgr2=0x55673993ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x55673993ae90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00594c0 tx=0x7fe7f005d680 comp rx=0 tx=0).stop 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 shutdown_connections 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x55673992ad70 0x55673992b150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739947240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739913c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 --2- 172.21.15.111:0/243767736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x55673993ae90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe7dd7fa640 0 client.6081 ms_handle_reset on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:14.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 >> 172.21.15.111:0/243767736 conn(0x5567398f9480 msgr2=0x5567398fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:14.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 shutdown_connections 2023-12-04T05:41:14.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 1 -- 172.21.15.111:0/243767736 wait complete. 2023-12-04T05:41:14.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.086+0000 7fe8075f9740 20 client.6081 trim_cache size 0 max 16384 2023-12-04T05:41:14.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.101+0000 7fe8075f9740 1 Processor -- start 2023-12-04T05:41:14.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 -- start start 2023-12-04T05:41:14.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x556739943fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739911a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739913d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567397e4270 con 0x556739925fa0 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739966ab0 con 0x55673993e360 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x556739809740 con 0x556739926380 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x556739943fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x556739943fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46444/0 (socket says 172.21.15.111:46444) 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.102+0000 7fe7fe59c640 1 -- 172.21.15.111:0/2506976431 learned_addr learned my addr 172.21.15.111:0/2506976431 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7fe59c640 1 -- 172.21.15.111:0/2506976431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 msgr2=0x556739911a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739911a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7fe59c640 1 -- 172.21.15.111:0/2506976431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 msgr2=0x556739913d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739913d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7fe59c640 1 -- 172.21.15.111:0/2506976431 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556739800d00 con 0x556739925fa0 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x556739943fe0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00619f0 tx=0x7fe7f005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b077622a1a5e98 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7ecff9640 1 -- 172.21.15.111:0/2506976431 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0058e90 con 0x556739925fa0 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7ecff9640 1 -- 172.21.15.111:0/2506976431 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0059300 con 0x556739925fa0 2023-12-04T05:41:14.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe7ecff9640 1 -- 172.21.15.111:0/2506976431 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0055320 con 0x556739925fa0 2023-12-04T05:41:14.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 -- 172.21.15.111:0/2506976431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 msgr2=0x556739943fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x556739943fe0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00619f0 tx=0x7fe7f005c000 comp rx=0 tx=0).stop 2023-12-04T05:41:14.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 -- 172.21.15.111:0/2506976431 shutdown_connections 2023-12-04T05:41:14.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55673993e360 0x556739913d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739926380 0x556739911a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 --2- 172.21.15.111:0/2506976431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556739925fa0 0x556739943fe0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 -- 172.21.15.111:0/2506976431 >> 172.21.15.111:0/2506976431 conn(0x5567398f9480 msgr2=0x5567398ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 -- 172.21.15.111:0/2506976431 shutdown_connections 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.103+0000 7fe8075f9740 1 -- 172.21.15.111:0/2506976431 wait complete. 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.104+0000 7fe8075f9740 1 -- start start 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.104+0000 7fe8075f9740 1 Processor -- start 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739925fa0 0x5567398fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739926380 0x5567398fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 0x5567399ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x556739929590 con 0x55673993e360 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567399299f0 con 0x556739926380 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55673992cce0 con 0x556739925fa0 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe8075f9740 10 client.0 _ll_register_callbacks cb 0x7fe7f0059d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-04T05:41:14.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 0x5567399ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 0x5567399ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46448/0 (socket says 172.21.15.111:46448) 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 -- 172.21.15.111:0/3895203817 learned_addr learned my addr 172.21.15.111:0/3895203817 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739925fa0 0x5567398fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 -- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739925fa0 msgr2=0x5567398fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739925fa0 0x5567398fcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fdd9b640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739926380 0x5567398fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 -- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739926380 msgr2=0x5567398fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739926380 0x5567398fcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 -- 172.21.15.111:0/3895203817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567398631f0 con 0x55673993e360 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7fed9d640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 0x5567399ae4c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8001930 tx=0x7fe7f8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b4dd8167a3b317 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.105+0000 7fe7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/3895203817 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80049e0 con 0x55673993e360 2023-12-04T05:41:14.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/3895203817 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f8004b40 con 0x55673993e360 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7f40237c0 con 0x55673993e360 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/3895203817 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f8001640 con 0x55673993e360 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 10 client.6087 fetch_fsmap learned FSMap version 23 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5567398264f0 con 0x55673993e360 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/3895203817 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f40237c0 con 0x55673993e360 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/3895203817 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 23) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe7f8004d20 con 0x55673993e360 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 10 client.6087 fetch_fsmap finished waiting for FSMap version 23 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 10 client.6087 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 20 client.6087 populate_metadata read hostname 'smithi111' 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x5567399aec90 0x556739955a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 4 client.6087 mds_command: new command op to 5868 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 --> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5567395656c0 con 0x5567399aec90 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x5567399aec90 0x556739955a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:41:14.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7dd7fa640 10 client.6087 ms_handle_connect on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.106+0000 7fe7fe59c640 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x5567399aec90 0x556739955a00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c5dd9043979cd server_cookie=7b452f528938ac09 in_seq=0 out_seq=0 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe7dd7fa640 1 -- 172.21.15.111:0/3895203817 <== mds.0 v2:172.21.15.111:6835/2442390644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x5567395656c0 con 0x5567399aec90 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe7dd7fa640 10 client.6087 handle_command_reply: tid=0 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 client.6087 shutdown 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 msgr2=0x5567399ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 0x5567399ae4c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8001930 tx=0x7fe7f8001a00 comp rx=0 tx=0).stop 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 shutdown_connections 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:6835/2442390644,v1:172.21.15.111:6837/2442390644] conn(0x5567399aec90 0x556739955a00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55673993e360 0x5567399ae4c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556739926380 0x5567398fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 --2- 172.21.15.111:0/3895203817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556739925fa0 0x5567398fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe7dd7fa640 0 client.6087 ms_handle_reset on v2:172.21.15.111:6835/2442390644 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 >> 172.21.15.111:0/3895203817 conn(0x5567398f9480 msgr2=0x556739944150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:14.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 shutdown_connections 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 1 -- 172.21.15.111:0/3895203817 wait complete. 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe8075f9740 20 client.6087 trim_cache size 0 max 16384 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "9686316e-a01b-4981-b0b7-e21bdb185d5e", 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5868, 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-12-04T05:41:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 10.795993786, 2023-12-04T05:41:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 23, 2023-12-04T05:41:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 45, 2023-12-04T05:41:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 45, 2023-12-04T05:41:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 15.85519407 2023-12-04T05:41:14.116 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7e006b2a0 msgr2=0x7fe7e006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe804989640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7e006b2a0 0x7fe7e006d760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe8000668f0 tx=0x7fe7f4008040 comp rx=0 tx=0).stop 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 msgr2=0x7fe800152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.107+0000 7fe804989640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe800152760 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0011e30 tx=0x7fe7f000f580 comp rx=0 tx=0).stop 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 shutdown_connections 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe7e006b2a0 0x7fe7e006d760 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8001516d0 0x7fe800151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe80015dbb0 0x7fe8000cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 --2- 172.21.15.111:0/33113854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7e8004d00 0x7fe800152760 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 >> 172.21.15.111:0/33113854 conn(0x7fe80009c4e0 msgr2=0x7fe8000d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 shutdown_connections 2023-12-04T05:41:14.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:41:14.108+0000 7fe804989640 1 -- 172.21.15.111:0/33113854 wait complete. 2023-12-04T05:41:14.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:14.119 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir {d0,d1} && touch {d0,d1}/file') 2023-12-04T05:41:14.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:14.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2023-12-04T05:41:14.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:14.221 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2023-12-04T05:41:14.278 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-12-04T05:41:14.431 INFO:teuthology.orchestra.run.smithi111.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-12-04T05:41:14.431 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-12-04T05:41:14.432 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2023-12-04T05:41:16.433 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-12-04T05:41:16.559 INFO:teuthology.orchestra.run.smithi111.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-12-04T05:41:16.559 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-12-04T05:41:16.559 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2023-12-04T05:41:16.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:41:16.622 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:41:18.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-12-04T05:41:18.687 INFO:teuthology.orchestra.run.smithi111.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-12-04T05:41:18.687 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-12-04T05:41:18.687 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2023-12-04T05:41:20.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-12-04T05:41:20.813 INFO:teuthology.orchestra.run.smithi111.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"17","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-12-04T05:41:20.814 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "17", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-12-04T05:41:20.814 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2023-12-04T05:41:22.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-12-04T05:41:22.941 INFO:teuthology.orchestra.run.smithi111.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"17","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-12-04T05:41:22.941 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "17", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-12-04T05:41:22.942 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2023-12-04T05:41:24.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2023-12-04T05:41:25.068 INFO:teuthology.orchestra.run.smithi111.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"18","version":"18","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2023-12-04T05:41:25.069 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "18", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "18" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2023-12-04T05:41:25.069 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d1', 1)] =?= [('/d0', 0), ('/d1', 1)] 2023-12-04T05:41:25.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:25.069 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d0/f0) 2023-12-04T05:41:25.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:41:25.088 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d1/f0) 2023-12-04T05:41:25.146 INFO:tasks.ceph.mds.a.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fa7da946640 time 2023-12-04T05:41:25.145815+0000 2023-12-04T05:41:25.146 INFO:tasks.ceph.mds.a.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:25.147 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.148 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa7e056eef8] 2023-12-04T05:41:25.148 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.148 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.148 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.148 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.148 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.149 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr:*** Caught signal (Aborted) ** 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr: in thread 7fa7da946640 thread_name:ms_dispatch 2023-12-04T05:41:25.150 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-12-04T05:41:25.145+0000 7fa7da946640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fa7da946640 time 2023-12-04T05:41:25.145815+0000 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa7e056eef8] 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.151 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.152 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa7dfc54db0] 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa7dfca154c] 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: raise() 2023-12-04T05:41:25.153 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: abort() 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa7e056ef52] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.154 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.155 INFO:tasks.ceph.mds.a.smithi111.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-12-04T05:41:25.146+0000 7fa7da946640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: in thread 7fa7da946640 thread_name:ms_dispatch 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa7dfc54db0] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa7dfca154c] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: raise() 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: abort() 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa7e056ef52] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.156 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.157 INFO:tasks.ceph.mds.a.smithi111.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:41:25.158 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.158 INFO:tasks.ceph.mds.a.smithi111.stderr: -1> 2023-12-04T05:41:25.145+0000 7fa7da946640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fa7da946640 time 2023-12-04T05:41:25.145815+0000 2023-12-04T05:41:25.158 INFO:tasks.ceph.mds.a.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:25.158 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.158 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.158 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa7e056eef8] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.159 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 0> 2023-12-04T05:41:25.146+0000 7fa7da946640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: in thread 7fa7da946640 thread_name:ms_dispatch 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa7dfc54db0] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa7dfca154c] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: raise() 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: abort() 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa7e056ef52] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.160 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.161 INFO:tasks.ceph.mds.a.smithi111.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: -1> 2023-12-04T05:41:25.145+0000 7fa7da946640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7fa7da946640 time 2023-12-04T05:41:25.145815+0000 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa7e056eef8] 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.162 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.163 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 0> 2023-12-04T05:41:25.146+0000 7fa7da946640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: in thread 7fa7da946640 thread_name:ms_dispatch 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:25.164 INFO:tasks.ceph.mds.a.smithi111.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa7dfc54db0] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa7dfca154c] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 3: raise() 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 4: abort() 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa7e056ef52] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7fa7e056f0b6] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55a92b5f7329] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55a92b5ec5b5] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55a92b59b85b] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55a92b4d5521] 2023-12-04T05:41:25.165 INFO:tasks.ceph.mds.a.smithi111.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55a92b4dc508] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55a92b4d1edd] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55a92b4d27bc] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a92b4be467] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a92b44248b] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a92b44297c] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a92b42c4b5] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7fa7e076040a] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7fa7e07f24a1] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7fa7dfc9f802] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7fa7dfc3f450] 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:41:25.166 INFO:tasks.ceph.mds.a.smithi111.stderr: 2023-12-04T05:41:25.667 INFO:tasks.ceph.mds.a.smithi111.stderr:daemon-helper: command crashed with signal 6 2023-12-04T05:41:29.660 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-04T05:41:36.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-12-04T05:41:42.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-12-04T05:41:46.236 INFO:tasks.ceph.mds.d.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f420aaec640 time 2023-12-04T05:41:46.235749+0000 2023-12-04T05:41:46.237 INFO:tasks.ceph.mds.d.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:46.237 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f421056eef8] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.238 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr:2023-12-04T05:41:46.235+0000 7f420aaec640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f420aaec640 time 2023-12-04T05:41:46.235749+0000 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f421056eef8] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.239 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.240 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr:*** Caught signal (Aborted) ** 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: in thread 7f420aaec640 thread_name:ms_dispatch 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f420fc54db0] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f420fca154c] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: raise() 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: abort() 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f421056ef52] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.241 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr:2023-12-04T05:41:46.236+0000 7f420aaec640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: in thread 7f420aaec640 thread_name:ms_dispatch 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.242 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f420fc54db0] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f420fca154c] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: raise() 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: abort() 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f421056ef52] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.243 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:41:46.244 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: -1> 2023-12-04T05:41:46.235+0000 7f420aaec640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f420aaec640 time 2023-12-04T05:41:46.235749+0000 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f421056eef8] 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.245 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.246 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 0> 2023-12-04T05:41:46.236+0000 7f420aaec640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: in thread 7f420aaec640 thread_name:ms_dispatch 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f420fc54db0] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f420fca154c] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: raise() 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: abort() 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f421056ef52] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.247 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.248 INFO:tasks.ceph.mds.d.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: -1> 2023-12-04T05:41:46.235+0000 7f420aaec640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f420aaec640 time 2023-12-04T05:41:46.235749+0000 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f421056eef8] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.249 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.250 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 0> 2023-12-04T05:41:46.236+0000 7f420aaec640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: in thread 7f420aaec640 thread_name:ms_dispatch 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f420fc54db0] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f420fca154c] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 3: raise() 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 4: abort() 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f421056ef52] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f421056f0b6] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x5561da63d329] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x5561da6325b5] 2023-12-04T05:41:46.251 INFO:tasks.ceph.mds.d.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x5561da5e185b] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x5561da51b521] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x5561da522508] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x5561da517edd] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x5561da5187bc] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5561da504467] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5561da48848b] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5561da48897c] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5561da4724b5] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f421076040a] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f42107f24a1] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f420fc9f802] 2023-12-04T05:41:46.252 INFO:tasks.ceph.mds.d.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f420fc3f450] 2023-12-04T05:41:46.253 INFO:tasks.ceph.mds.d.smithi192.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:41:46.253 INFO:tasks.ceph.mds.d.smithi192.stderr: 2023-12-04T05:41:46.506 INFO:tasks.ceph.mds.d.smithi192.stderr:daemon-helper: command crashed with signal 6 2023-12-04T05:41:46.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:41:46.685 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:41:48.576 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:41:48.575+0000 7f344645d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-04T05:41:49.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-12-04T05:41:49.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-12-04T05:41:55.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~26s 2023-12-04T05:41:55.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2023-12-04T05:42:02.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~33s 2023-12-04T05:42:02.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2023-12-04T05:42:06.247 INFO:tasks.ceph.mds.b.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f0e158a1640 time 2023-12-04T05:42:06.247250+0000 2023-12-04T05:42:06.247 INFO:tasks.ceph.mds.b.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:42:06.248 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.248 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0e1b36eef8] 2023-12-04T05:42:06.248 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.248 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.249 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr:2023-12-04T05:42:06.247+0000 7f0e158a1640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f0e158a1640 time 2023-12-04T05:42:06.247250+0000 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0e1b36eef8] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.250 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr:*** Caught signal (Aborted) ** 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: in thread 7f0e158a1640 thread_name:ms_dispatch 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.251 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0e1aa54db0] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0e1aaa154c] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: raise() 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: abort() 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0e1b36ef52] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.252 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.253 INFO:tasks.ceph.mds.b.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr:2023-12-04T05:42:06.248+0000 7f0e158a1640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: in thread 7f0e158a1640 thread_name:ms_dispatch 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0e1aa54db0] 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0e1aaa154c] 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: raise() 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: abort() 2023-12-04T05:42:06.254 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0e1b36ef52] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.255 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: -1> 2023-12-04T05:42:06.247+0000 7f0e158a1640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f0e158a1640 time 2023-12-04T05:42:06.247250+0000 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.256 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0e1b36eef8] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.257 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 0> 2023-12-04T05:42:06.248+0000 7f0e158a1640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: in thread 7f0e158a1640 thread_name:ms_dispatch 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0e1aa54db0] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0e1aaa154c] 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: raise() 2023-12-04T05:42:06.258 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: abort() 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0e1b36ef52] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.259 INFO:tasks.ceph.mds.b.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: -1> 2023-12-04T05:42:06.247+0000 7f0e158a1640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: In function 'bool Locker::xlock_start(SimpleLock*, MDRequestRef&)' thread 7f0e158a1640 time 2023-12-04T05:42:06.247250+0000 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7531-g47c1c547/rpm/el9/BUILD/ceph-18.0.0-7531-g47c1c547/src/mds/Locker.cc: 1963: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.260 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0e1b36eef8] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.261 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 0> 2023-12-04T05:42:06.248+0000 7f0e158a1640 -1 *** Caught signal (Aborted) ** 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: in thread 7f0e158a1640 thread_name:ms_dispatch 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: ceph version 18.0.0-7531-g47c1c547 (47c1c547372ed5bff7984647dc200bac0e019caf) reef (dev) 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0e1aa54db0] 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0e1aaa154c] 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 3: raise() 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 4: abort() 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0e1b36ef52] 2023-12-04T05:42:06.262 INFO:tasks.ceph.mds.b.smithi192.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f0b6) [0x7f0e1b36f0b6] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 7: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x939) [0x55ab5e73a329] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 8: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x20f5) [0x55ab5e72f5b5] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 9: (MDCache::path_traverse(boost::intrusive_ptr&, ContextFactory&, filepath const&, int, std::vector >*, CInode**)+0x79b) [0x55ab5e6de85b] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 10: (Server::rdlock_path_xlock_dentry(boost::intrusive_ptr&, bool, bool, bool, bool)+0x371) [0x55ab5e618521] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 11: (Server::handle_client_openc(boost::intrusive_ptr&)+0xd8) [0x55ab5e61f508] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 12: (Server::handle_peer_auth_pin_ack(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x83d) [0x55ab5e614edd] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 13: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0x6bc) [0x55ab5e6157bc] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ab5e601467] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55ab5e58548b] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ab5e58597c] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55ab5e56f4b5] 2023-12-04T05:42:06.263 INFO:tasks.ceph.mds.b.smithi192.stderr: 18: (DispatchQueue::entry()+0x53a) [0x7f0e1b56040a] 2023-12-04T05:42:06.264 INFO:tasks.ceph.mds.b.smithi192.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x3f24a1) [0x7f0e1b5f24a1] 2023-12-04T05:42:06.265 INFO:tasks.ceph.mds.b.smithi192.stderr: 20: /lib64/libc.so.6(+0x9f802) [0x7f0e1aa9f802] 2023-12-04T05:42:06.265 INFO:tasks.ceph.mds.b.smithi192.stderr: 21: /lib64/libc.so.6(+0x3f450) [0x7f0e1aa3f450] 2023-12-04T05:42:06.266 INFO:tasks.ceph.mds.b.smithi192.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-12-04T05:42:06.266 INFO:tasks.ceph.mds.b.smithi192.stderr: 2023-12-04T05:42:06.534 INFO:tasks.ceph.mds.b.smithi192.stderr:daemon-helper: command crashed with signal 6 2023-12-04T05:42:08.577 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:42:08.576+0000 7f344645d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-04T05:42:08.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~39s 2023-12-04T05:42:08.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-12-04T05:42:08.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~20s 2023-12-04T05:42:15.244 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~46s 2023-12-04T05:42:15.244 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~6s 2023-12-04T05:42:15.244 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~26s 2023-12-04T05:42:16.864 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:42:16.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:42:21.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~52s 2023-12-04T05:42:21.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-12-04T05:42:21.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~32s 2023-12-04T05:42:28.067 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~59s 2023-12-04T05:42:28.067 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~19s 2023-12-04T05:42:28.067 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~39s 2023-12-04T05:42:34.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~65s 2023-12-04T05:42:34.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~26s 2023-12-04T05:42:34.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~45s 2023-12-04T05:42:40.890 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~71s 2023-12-04T05:42:40.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~32s 2023-12-04T05:42:40.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~52s 2023-12-04T05:42:47.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:42:47.022 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:42:47.305 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~78s 2023-12-04T05:42:47.305 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~38s 2023-12-04T05:42:47.305 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~58s 2023-12-04T05:42:53.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~84s 2023-12-04T05:42:53.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~45s 2023-12-04T05:42:53.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~64s 2023-12-04T05:43:00.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~91s 2023-12-04T05:43:00.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~51s 2023-12-04T05:43:00.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~71s 2023-12-04T05:43:06.541 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~97s 2023-12-04T05:43:06.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~58s 2023-12-04T05:43:06.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~77s 2023-12-04T05:43:12.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~103s 2023-12-04T05:43:12.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~64s 2023-12-04T05:43:12.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~84s 2023-12-04T05:43:17.283 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:43:17.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:43:19.365 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~110s 2023-12-04T05:43:19.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~71s 2023-12-04T05:43:19.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~90s 2023-12-04T05:43:25.777 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~116s 2023-12-04T05:43:25.777 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~77s 2023-12-04T05:43:25.777 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~96s 2023-12-04T05:43:32.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~123s 2023-12-04T05:43:32.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~83s 2023-12-04T05:43:32.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~103s 2023-12-04T05:43:38.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~129s 2023-12-04T05:43:38.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~90s 2023-12-04T05:43:38.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~109s 2023-12-04T05:43:45.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~136s 2023-12-04T05:43:45.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~96s 2023-12-04T05:43:45.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~116s 2023-12-04T05:43:47.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:43:47.425 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:43:51.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~142s 2023-12-04T05:43:51.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~103s 2023-12-04T05:43:51.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~122s 2023-12-04T05:43:57.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~148s 2023-12-04T05:43:57.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~109s 2023-12-04T05:43:57.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~129s 2023-12-04T05:44:04.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~155s 2023-12-04T05:44:04.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~115s 2023-12-04T05:44:04.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~135s 2023-12-04T05:44:10.661 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~161s 2023-12-04T05:44:10.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~122s 2023-12-04T05:44:10.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~141s 2023-12-04T05:44:17.070 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~168s 2023-12-04T05:44:17.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~128s 2023-12-04T05:44:17.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~148s 2023-12-04T05:44:17.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:44:17.539 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:44:23.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~174s 2023-12-04T05:44:23.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~135s 2023-12-04T05:44:23.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~154s 2023-12-04T05:44:29.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~180s 2023-12-04T05:44:29.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~141s 2023-12-04T05:44:29.895 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~161s 2023-12-04T05:44:36.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~187s 2023-12-04T05:44:36.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~147s 2023-12-04T05:44:36.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~167s 2023-12-04T05:44:42.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~193s 2023-12-04T05:44:42.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~154s 2023-12-04T05:44:42.718 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~173s 2023-12-04T05:44:47.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:44:47.677 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:44:49.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~200s 2023-12-04T05:44:49.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~160s 2023-12-04T05:44:49.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~180s 2023-12-04T05:44:55.539 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~206s 2023-12-04T05:44:55.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~167s 2023-12-04T05:44:55.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~186s 2023-12-04T05:45:01.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~212s 2023-12-04T05:45:01.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~173s 2023-12-04T05:45:01.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~193s 2023-12-04T05:45:08.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~219s 2023-12-04T05:45:08.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~180s 2023-12-04T05:45:08.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~199s 2023-12-04T05:45:14.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~225s 2023-12-04T05:45:14.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~186s 2023-12-04T05:45:14.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~205s 2023-12-04T05:45:17.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:45:17.809 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:45:21.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~232s 2023-12-04T05:45:21.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~192s 2023-12-04T05:45:21.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~212s 2023-12-04T05:45:27.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~238s 2023-12-04T05:45:27.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~199s 2023-12-04T05:45:27.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~218s 2023-12-04T05:45:34.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~245s 2023-12-04T05:45:34.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~205s 2023-12-04T05:45:34.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~225s 2023-12-04T05:45:40.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~251s 2023-12-04T05:45:40.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~212s 2023-12-04T05:45:40.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~231s 2023-12-04T05:45:46.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~257s 2023-12-04T05:45:46.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~218s 2023-12-04T05:45:46.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~238s 2023-12-04T05:45:47.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:45:47.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:45:53.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~264s 2023-12-04T05:45:53.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~224s 2023-12-04T05:45:53.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~244s 2023-12-04T05:45:59.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~270s 2023-12-04T05:45:59.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~231s 2023-12-04T05:45:59.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~250s 2023-12-04T05:46:06.055 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~277s 2023-12-04T05:46:06.055 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~237s 2023-12-04T05:46:06.055 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~257s 2023-12-04T05:46:12.464 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~283s 2023-12-04T05:46:12.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~244s 2023-12-04T05:46:12.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~263s 2023-12-04T05:46:18.100 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:46:18.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:46:18.876 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~289s 2023-12-04T05:46:18.876 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~250s 2023-12-04T05:46:18.876 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~270s 2023-12-04T05:46:25.137 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2023-12-04T05:46:25.138 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2023-12-04T05:46:25.288 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~296s 2023-12-04T05:46:25.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~256s 2023-12-04T05:46:25.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~276s 2023-12-04T05:46:31.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~302s 2023-12-04T05:46:31.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~263s 2023-12-04T05:46:31.701 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~282s 2023-12-04T05:46:31.701 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-12-04T05:46:31.701 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:46:31.701 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:31.721 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:31.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:46:31.764 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-04T05:46:31.764 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:46:31.811 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-04T05:46:31.858 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-04T05:46:31.858 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:46:31.858 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:31.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.869+0000 7fe357ee0640 1 Processor -- start 2023-12-04T05:46:31.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.869+0000 7fe357ee0640 1 -- start start 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe357ee0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe35013ced0 0x7fe35015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe357ee0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35015dbf0 con 0x7fe35015a600 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe357ee0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35015eb80 con 0x7fe35006dc40 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe357ee0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35015b170 con 0x7fe35013ced0 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe35547b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe35015a600 0x7fe35006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38400/0 (socket says 172.21.15.111:38400) 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe35547b640 1 -- 172.21.15.111:0/4166468077 learned_addr learned my addr 172.21.15.111:0/4166468077 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe355c7c640 1 --2- 172.21.15.111:0/4166468077 >> v2:172.21.15.192:3300/0 conn(0x7fe35013ced0 0x7fe35015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115710640 0 0) 0x7fe35015dbf0 con 0x7fe35015a600 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.870+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe33c003520 con 0x7fe35015a600 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1066798377 0 0) 0x7fe33c003520 con 0x7fe35015a600 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 >> v2:172.21.15.192:3300/0 conn(0x7fe35013ced0 msgr2=0x7fe35015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:31.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 --2- 172.21.15.111:0/4166468077 >> v2:172.21.15.192:3300/0 conn(0x7fe35013ced0 0x7fe35015e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 >> v1:172.21.15.192:6789/0 conn(0x7fe35006dc40 legacy=0x7fe3500cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35015b450 con 0x7fe35015a600 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fe340001670 con 0x7fe35015a600 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 >> v1:172.21.15.111:6789/0 conn(0x7fe35015a600 legacy=0x7fe35006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe33c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c0047c0 0x7fe33c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe33c005180 0x7fe33c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35015b170 con 0x7fe33c003970 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c00c020 con 0x7fe33c0047c0 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c00c1e0 con 0x7fe33c005180 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe355c7c640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe33c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe35647d640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe33c005180 0x7fe33c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe35547b640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c0047c0 0x7fe33c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe355c7c640 1 -- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe33c005180 msgr2=0x7fe33c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe355c7c640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe33c005180 0x7fe33c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe355c7c640 1 -- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c0047c0 msgr2=0x7fe33c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe355c7c640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c0047c0 0x7fe33c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe355c7c640 1 -- 172.21.15.111:0/4166468077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c003520 con 0x7fe33c003970 2023-12-04T05:46:31.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe35647d640 1 -- 172.21.15.111:0/4166468077 reap_dead start 2023-12-04T05:46:31.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.871+0000 7fe35547b640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe33c0047c0 0x7fe33c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:31.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.872+0000 7fe355c7c640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe33c003d90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009ac0 tx=0x7fe33c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ed91b5bc40110f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.872+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c02c6b0 con 0x7fe33c003970 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.872+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c02c810 con 0x7fe33c003970 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.872+0000 7fe354c7a640 1 -- 172.21.15.111:0/4166468077 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c02cba0 con 0x7fe33c003970 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 -- 172.21.15.111:0/4166468077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 msgr2=0x7fe33c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe33c003d90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009ac0 tx=0x7fe33c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 -- 172.21.15.111:0/4166468077 shutdown_connections 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 --2- 172.21.15.111:0/4166468077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe33c003d90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 -- 172.21.15.111:0/4166468077 >> 172.21.15.111:0/4166468077 conn(0x7fe3500662b0 msgr2=0x7fe350066690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 -- 172.21.15.111:0/4166468077 shutdown_connections 2023-12-04T05:46:31.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.873+0000 7fe357ee0640 1 -- 172.21.15.111:0/4166468077 wait complete. 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.874+0000 7fe357ee0640 1 Processor -- start 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.874+0000 7fe357ee0640 1 -- start start 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe35014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe35006a730 0x7fe3501357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe35006abb0 0x7fe350135d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501353c0 con 0x7fe33c003970 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501344f0 con 0x7fe35006abb0 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe350134740 con 0x7fe35006a730 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe35014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe35014f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45628/0 (socket says 172.21.15.111:45628) 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 -- 172.21.15.111:0/2584010333 learned_addr learned my addr 172.21.15.111:0/2584010333 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe35547b640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe35006a730 0x7fe3501357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe35647d640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe35006abb0 0x7fe350135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 -- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe35006a730 msgr2=0x7fe3501357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe35006a730 0x7fe3501357e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 -- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe35006abb0 msgr2=0x7fe350135d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe35006abb0 0x7fe350135d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 -- 172.21.15.111:0/2584010333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350134a20 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe355c7c640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe35014f260 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe34c037040 tx=0x7fe34c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003810 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c003970 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe350134d00 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003d20 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.875+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35014e200 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.876+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe350067460 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.877+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe34c034440 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.877+0000 7fe3467fc640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe33406b2d0 0x7fe33406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.877+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe34c0abb30 con 0x7fe33c003970 2023-12-04T05:46:31.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.878+0000 7fe35547b640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe33406b2d0 0x7fe33406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.878+0000 7fe35547b640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe33406b2d0 0x7fe33406d790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe35015d9a0 tx=0x7fe340009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:31.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.882+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe34c076a80 con 0x7fe33c003970 2023-12-04T05:46:31.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.980+0000 7f3512f68640 1 Processor -- start 2023-12-04T05:46:31.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.980+0000 7f3512f68640 1 -- start start 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f3512f68640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f3512f68640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0cdcb0 con 0x7f350c0b2ad0 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f3512f68640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0d39e0 con 0x7f350c0d1780 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f3512f68640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0d3cc0 con 0x7f350c0b20a0 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f3511505640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f350c0b2ad0 0x7f350c0ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38414/0 (socket says 172.21.15.111:38414) 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f3511505640 1 -- 172.21.15.111:0/1436736746 learned_addr learned my addr 172.21.15.111:0/1436736746 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:31.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f350bfff640 1 --2- 172.21.15.111:0/1436736746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f350b7fe640 1 -- 172.21.15.111:0/1436736746 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033090335 0 0) 0x7f350c0cdcb0 con 0x7f350c0b2ad0 2023-12-04T05:46:31.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f350b7fe640 1 -- 172.21.15.111:0/1436736746 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34fc003520 con 0x7f350c0b2ad0 2023-12-04T05:46:31.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f350bfff640 1 -- 172.21.15.111:0/1436736746 >> v1:172.21.15.192:6789/0 conn(0x7f350c0d1780 legacy=0x7f350c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f350bfff640 1 -- 172.21.15.111:0/1436736746 >> v1:172.21.15.111:6789/0 conn(0x7f350c0b2ad0 legacy=0x7f350c0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.981+0000 7f350bfff640 1 -- 172.21.15.111:0/1436736746 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0cdf50 con 0x7f350c0b20a0 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.982+0000 7f350bfff640 1 --2- 172.21.15.111:0/1436736746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c0d01e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34f80097d0 tx=0x7f34f802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5593f83174c87130 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.982+0000 7f350b7fe640 1 -- 172.21.15.111:0/1436736746 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8031800 con 0x7f350c0b20a0 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f350b7fe640 1 -- 172.21.15.111:0/1436736746 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f8031960 con 0x7f350c0b20a0 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f350b7fe640 1 -- 172.21.15.111:0/1436736746 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8031cd0 con 0x7f350c0b20a0 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 -- 172.21.15.111:0/1436736746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 msgr2=0x7f350c0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 --2- 172.21.15.111:0/1436736746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c0d01e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34f80097d0 tx=0x7f34f802b8a0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 -- 172.21.15.111:0/1436736746 shutdown_connections 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 --2- 172.21.15.111:0/1436736746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c0d01e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 -- 172.21.15.111:0/1436736746 >> 172.21.15.111:0/1436736746 conn(0x7f350c09c4e0 msgr2=0x7f350c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:31.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 -- 172.21.15.111:0/1436736746 shutdown_connections 2023-12-04T05:46:31.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.983+0000 7f3512f68640 1 -- 172.21.15.111:0/1436736746 wait complete. 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.984+0000 7f3512f68640 1 Processor -- start 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.984+0000 7f3512f68640 1 -- start start 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3512f68640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c1f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3512f68640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c0b2ad0 0x7f350c1f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3512f68640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 0x7f350c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3512f68640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b6f00 con 0x7f350c0d1780 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3512f68640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b5e60 con 0x7f350c0b2ad0 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3512f68640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b6140 con 0x7f350c0b20a0 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3511505640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 0x7f350c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3511505640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 0x7f350c06c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45634/0 (socket says 172.21.15.111:45634) 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3511505640 1 -- 172.21.15.111:0/675856909 learned_addr learned my addr 172.21.15.111:0/675856909 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f3510d04640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c1f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.985+0000 7f350bfff640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c0b2ad0 0x7f350c1f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3511505640 1 -- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 msgr2=0x7f350c1f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3511505640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c1f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3511505640 1 -- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c0b2ad0 msgr2=0x7f350c1f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3511505640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c0b2ad0 0x7f350c1f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3511505640 1 -- 172.21.15.111:0/675856909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8009400 con 0x7f350c0d1780 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3511505640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 0x7f350c06c4e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f350400cb40 tx=0x7f350400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400aca0 con 0x7f350c0d1780 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350400ae00 con 0x7f350c0d1780 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504012620 con 0x7f350c0d1780 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c0b6420 con 0x7f350c0d1780 2023-12-04T05:46:31.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.986+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c0b69e0 con 0x7f350c0d1780 2023-12-04T05:46:31.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.987+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c0693d0 con 0x7f350c0d1780 2023-12-04T05:46:31.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.988+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f3504007070 con 0x7f350c0d1780 2023-12-04T05:46:31.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.988+0000 7f35097fa640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f34f006b2a0 0x7f34f006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:31.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.988+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3504089c60 con 0x7f350c0d1780 2023-12-04T05:46:31.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.988+0000 7f3510d04640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f34f006b2a0 0x7f34f006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:31.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.988+0000 7f3510d04640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f34f006b2a0 0x7f34f006d760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f35000016e0 tx=0x7f3500009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:31.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:31.992+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3504054a90 con 0x7f350c0d1780 2023-12-04T05:46:32.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.059+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3500b3210 con 0x7fe33c003970 2023-12-04T05:46:32.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.060+0000 7fe3467fc640 1 -- 172.21.15.111:0/2584010333 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7fe34c07a0a0 con 0x7fe33c003970 2023-12-04T05:46:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:32.063 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:32.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe33406b2d0 msgr2=0x7fe33406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe33406b2d0 0x7fe33406d790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe35015d9a0 tx=0x7fe340009040 comp rx=0 tx=0).stop 2023-12-04T05:46:32.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 msgr2=0x7fe35014f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe35014f260 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe34c037040 tx=0x7fe34c004730 comp rx=0 tx=0).stop 2023-12-04T05:46:32.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 shutdown_connections 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe33406b2d0 0x7fe33406d790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe35006abb0 0x7fe350135d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe35006a730 0x7fe3501357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 --2- 172.21.15.111:0/2584010333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe33c003970 0x7fe35014f260 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 >> 172.21.15.111:0/2584010333 conn(0x7fe3500662b0 msgr2=0x7fe3500cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 shutdown_connections 2023-12-04T05:46:32.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.062+0000 7fe357ee0640 1 -- 172.21.15.111:0/2584010333 wait complete. 2023-12-04T05:46:32.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:41:00.843728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:41:01.612301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:46:32.076 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:46:32.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.167+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f350c0b6bd0 con 0x7f350c0d1780 2023-12-04T05:46:32.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.168+0000 7f35097fa640 1 -- 172.21.15.111:0/675856909 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f3504017020 con 0x7f350c0d1780 2023-12-04T05:46:32.170 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:32.170 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:32.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f34f006b2a0 msgr2=0x7f34f006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f34f006b2a0 0x7f34f006d760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f35000016e0 tx=0x7f3500009040 comp rx=0 tx=0).stop 2023-12-04T05:46:32.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 msgr2=0x7f350c06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 0x7f350c06c4e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f350400cb40 tx=0x7f350400a060 comp rx=0 tx=0).stop 2023-12-04T05:46:32.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 shutdown_connections 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f34f006b2a0 0x7f34f006d760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f350c0d1780 0x7f350c06c4e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c0b2ad0 0x7f350c1f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 --2- 172.21.15.111:0/675856909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f350c0b20a0 0x7f350c1f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 >> 172.21.15.111:0/675856909 conn(0x7f350c09c4e0 msgr2=0x7f350c0d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 shutdown_connections 2023-12-04T05:46:32.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.171+0000 7f3512f68640 1 -- 172.21.15.111:0/675856909 wait complete. 2023-12-04T05:46:32.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.178+0000 7fb724831640 1 Processor -- start 2023-12-04T05:46:32.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.178+0000 7fb724831640 1 -- start start 2023-12-04T05:46:32.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.179+0000 7fb724831640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb72015d480 0x7fb7200d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.179+0000 7fb724831640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb7200d1310 0x7fb7200cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.179+0000 7fb724831640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72015ddc0 con 0x7fb72015d860 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.179+0000 7fb724831640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72015e0a0 con 0x7fb7200d1310 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.179+0000 7fb724831640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72015e380 con 0x7fb72015d480 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb7200d1310 0x7fb7200cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb7200d1310 0x7fb7200cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45650/0 (socket says 172.21.15.111:45650) 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2598842629 learned_addr learned my addr 172.21.15.111:0/2598842629 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71e59c640 1 --2- 172.21.15.111:0/2598842629 >> v2:172.21.15.192:3300/0 conn(0x7fb72015d480 0x7fb7200d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2598842629 >> v2:172.21.15.192:3300/0 conn(0x7fb72015d480 msgr2=0x7fb7200d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 --2- 172.21.15.111:0/2598842629 >> v2:172.21.15.192:3300/0 conn(0x7fb72015d480 0x7fb7200d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2598842629 >> v1:172.21.15.192:6789/0 conn(0x7fb72015d860 legacy=0x7fb720160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2598842629 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72015e660 con 0x7fb7200d1310 2023-12-04T05:46:32.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.180+0000 7fb71e59c640 1 --2- 172.21.15.111:0/2598842629 >> v2:172.21.15.192:3300/0 conn(0x7fb72015d480 0x7fb7200d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:32.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.181+0000 7fb71dd9b640 1 --2- 172.21.15.111:0/2598842629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7200d1310 0x7fb7200cfad0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb7200b5d30 tx=0x7fb71402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b68b945033cc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.181+0000 7fb71d59a640 1 -- 172.21.15.111:0/2598842629 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714030860 con 0x7fb7200d1310 2023-12-04T05:46:32.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.181+0000 7fb71d59a640 1 -- 172.21.15.111:0/2598842629 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7140309c0 con 0x7fb7200d1310 2023-12-04T05:46:32.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.181+0000 7fb71d59a640 1 -- 172.21.15.111:0/2598842629 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714030d50 con 0x7fb7200d1310 2023-12-04T05:46:32.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:41:00.843728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:41:01.612301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:46:32.187 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.181+0000 7fb724831640 1 -- 172.21.15.111:0/2598842629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7200d1310 msgr2=0x7fb7200cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.181+0000 7fb724831640 1 --2- 172.21.15.111:0/2598842629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7200d1310 0x7fb7200cfad0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb7200b5d30 tx=0x7fb71402b9a0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.182+0000 7fb724831640 1 -- 172.21.15.111:0/2598842629 shutdown_connections 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.182+0000 7fb724831640 1 --2- 172.21.15.111:0/2598842629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7200d1310 0x7fb7200cfad0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.182+0000 7fb724831640 1 --2- 172.21.15.111:0/2598842629 >> v2:172.21.15.192:3300/0 conn(0x7fb72015d480 0x7fb7200d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.182+0000 7fb724831640 1 -- 172.21.15.111:0/2598842629 >> 172.21.15.111:0/2598842629 conn(0x7fb72009c540 msgr2=0x7fb7200d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.182+0000 7fb724831640 1 -- 172.21.15.111:0/2598842629 shutdown_connections 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.182+0000 7fb724831640 1 -- 172.21.15.111:0/2598842629 wait complete. 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.183+0000 7fb724831640 1 Processor -- start 2023-12-04T05:46:32.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.183+0000 7fb724831640 1 -- start start 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200d1310 0x7fb720148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 0x7fb72014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015d860 0x7fb7200d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201f0a40 con 0x7fb72015d480 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200c6e00 con 0x7fb7200d1310 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72014c620 con 0x7fb72015d860 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 0x7fb72014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 0x7fb72014c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45664/0 (socket says 172.21.15.111:45664) 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2559668122 learned_addr learned my addr 172.21.15.111:0/2559668122 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71ed9d640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015d860 0x7fb7200d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71e59c640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200d1310 0x7fb720148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015d860 msgr2=0x7fb7200d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015d860 0x7fb7200d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200d1310 msgr2=0x7fb720148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200d1310 0x7fb720148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 -- 172.21.15.111:0/2559668122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140094d0 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71e59c640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200d1310 0x7fb720148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb71dd9b640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 0x7fb72014c0e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb71402b9e0 tx=0x7fb714004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714030850 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb714043400 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7201f3530 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7140437b0 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.184+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7201f3af0 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.185+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb720067980 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.186+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb714043910 con 0x7fb72015d480 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.186+0000 7fb70effd640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb70006b2a0 0x7fb70006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.186+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb71403e070 con 0x7fb72015d480 2023-12-04T05:46:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.186+0000 7fb71e59c640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb70006b2a0 0x7fb70006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.187+0000 7fb71e59c640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb70006b2a0 0x7fb70006d760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb71000aa30 tx=0x7fb710008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.190+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb714077ea0 con 0x7fb72015d480 2023-12-04T05:46:32.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.294+0000 7fe4b54b4640 1 Processor -- start 2023-12-04T05:46:32.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.294+0000 7fe4b54b4640 1 -- start start 2023-12-04T05:46:32.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4b54b4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b0160e30 0x7fe4b00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4b54b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00b30e0 0x7fe4b006a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4b54b4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b00cfa10 con 0x7fe4b00cf0f0 2023-12-04T05:46:32.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4b54b4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b00b4700 con 0x7fe4b0160e30 2023-12-04T05:46:32.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4b54b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b01406f0 con 0x7fe4b00b30e0 2023-12-04T05:46:32.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b0160e30 0x7fe4b00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4b0160e30 0x7fe4b00cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45678/0 (socket says 172.21.15.111:45678) 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 -- 172.21.15.111:0/2735052568 learned_addr learned my addr 172.21.15.111:0/2735052568 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4af7fe640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00b30e0 0x7fe4b006a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 -- 172.21.15.111:0/2735052568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00b30e0 msgr2=0x7fe4b006a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00b30e0 0x7fe4b006a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 -- 172.21.15.111:0/2735052568 >> v1:172.21.15.192:6789/0 conn(0x7fe4b00cf0f0 legacy=0x7fe4b0143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4a7fff640 1 -- 172.21.15.111:0/2735052568 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b0161360 con 0x7fe4b0160e30 2023-12-04T05:46:32.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.295+0000 7fe4af7fe640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00b30e0 0x7fe4b006a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.296+0000 7fe4a7fff640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b00cf4d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0070010 tx=0x7fe49802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0f2340796e09e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.296+0000 7fe4ae7fc640 1 -- 172.21.15.111:0/2735052568 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498031830 con 0x7fe4b0160e30 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.296+0000 7fe4ae7fc640 1 -- 172.21.15.111:0/2735052568 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498031990 con 0x7fe4b0160e30 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.296+0000 7fe4ae7fc640 1 -- 172.21.15.111:0/2735052568 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498031d20 con 0x7fe4b0160e30 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 -- 172.21.15.111:0/2735052568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 msgr2=0x7fe4b00cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b00cf4d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0070010 tx=0x7fe49802b8d0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 -- 172.21.15.111:0/2735052568 shutdown_connections 2023-12-04T05:46:32.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00b30e0 0x7fe4b006a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/2735052568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b00cf4d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 -- 172.21.15.111:0/2735052568 >> 172.21.15.111:0/2735052568 conn(0x7fe4b00b1de0 msgr2=0x7fe4b00b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 -- 172.21.15.111:0/2735052568 shutdown_connections 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.297+0000 7fe4b54b4640 1 -- 172.21.15.111:0/2735052568 wait complete. 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.298+0000 7fe4b54b4640 1 Processor -- start 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.298+0000 7fe4b54b4640 1 -- start start 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b00b30e0 0x7fe4b01f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00cf0f0 0x7fe4b006a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b0143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b01f63b0 con 0x7fe4b0160e30 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b01f6690 con 0x7fe4b00b30e0 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b01f6970 con 0x7fe4b00cf0f0 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b0143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b0143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45694/0 (socket says 172.21.15.111:45694) 2023-12-04T05:46:32.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 -- 172.21.15.111:0/3381762269 learned_addr learned my addr 172.21.15.111:0/3381762269 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4aeffd640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b00b30e0 0x7fe4b01f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4a7fff640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00cf0f0 0x7fe4b006a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 -- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00cf0f0 msgr2=0x7fe4b006a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00cf0f0 0x7fe4b006a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 -- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b00b30e0 msgr2=0x7fe4b01f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b00b30e0 0x7fe4b01f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 -- 172.21.15.111:0/3381762269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980094d0 con 0x7fe4b0160e30 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4af7fe640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b0143f10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800cef0 tx=0x7fe4a800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800d840 con 0x7fe4b0160e30 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a800d9c0 con 0x7fe4b0160e30 2023-12-04T05:46:32.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b01f6c50 con 0x7fe4b0160e30 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800dd70 con 0x7fe4b0160e30 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.299+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b01f7210 con 0x7fe4b0160e30 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.300+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b014e4f0 con 0x7fe4b0160e30 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.301+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe4a801e3b0 con 0x7fe4b0160e30 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.301+0000 7fe4a77fe640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe49006b2a0 0x7fe49006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.301+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe4a808a6f0 con 0x7fe4b0160e30 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.301+0000 7fe4aeffd640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe49006b2a0 0x7fe49006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.302+0000 7fe4aeffd640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe49006b2a0 0x7fe49006d760 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0004f60 tx=0x7fe4a0007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.306+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4b014e4f0 con 0x7fe4b0160e30 2023-12-04T05:46:32.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.369+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7200c8a20 con 0x7fb72015d480 2023-12-04T05:46:32.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.370+0000 7fb70effd640 1 -- 172.21.15.111:0/2559668122 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7200c8a20 con 0x7fb72015d480 2023-12-04T05:46:32.371 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:46:32.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb70006b2a0 msgr2=0x7fb70006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb70006b2a0 0x7fb70006d760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb71000aa30 tx=0x7fb710008040 comp rx=0 tx=0).stop 2023-12-04T05:46:32.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 msgr2=0x7fb72014c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 0x7fb72014c0e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb71402b9e0 tx=0x7fb714004730 comp rx=0 tx=0).stop 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 shutdown_connections 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb70006b2a0 0x7fb70006d760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015d860 0x7fb7200d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb72015d480 0x7fb72014c0e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 --2- 172.21.15.111:0/2559668122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200d1310 0x7fb720148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 >> 172.21.15.111:0/2559668122 conn(0x7fb72009c540 msgr2=0x7fb7200d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 shutdown_connections 2023-12-04T05:46:32.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.372+0000 7fb724831640 1 -- 172.21.15.111:0/2559668122 wait complete. 2023-12-04T05:46:32.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:46:32.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.480+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4b014e4f0 con 0x7fe4b0160e30 2023-12-04T05:46:32.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.481+0000 7fe4a77fe640 1 -- 172.21.15.111:0/3381762269 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4a801a020 con 0x7fe4b0160e30 2023-12-04T05:46:32.483 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:46:32.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe49006b2a0 msgr2=0x7fe49006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe49006b2a0 0x7fe49006d760 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0004f60 tx=0x7fe4a0007e30 comp rx=0 tx=0).stop 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 msgr2=0x7fe4b0143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b0143f10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800cef0 tx=0x7fe4a800a710 comp rx=0 tx=0).stop 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 shutdown_connections 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe49006b2a0 0x7fe49006d760 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4b0160e30 0x7fe4b0143f10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b00cf0f0 0x7fe4b006a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 --2- 172.21.15.111:0/3381762269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b00b30e0 0x7fe4b01f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 >> 172.21.15.111:0/3381762269 conn(0x7fe4b00b1de0 msgr2=0x7fe4b00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 shutdown_connections 2023-12-04T05:46:32.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.483+0000 7fe4b54b4640 1 -- 172.21.15.111:0/3381762269 wait complete. 2023-12-04T05:46:32.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.487+0000 7f1d4246b640 1 Processor -- start 2023-12-04T05:46:32.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.487+0000 7f1d4246b640 1 -- start start 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.488+0000 7f1d4246b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c15e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.488+0000 7f1d4246b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d3c15e680 0x7f1d3c0b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.488+0000 7f1d4246b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c15dcb0 con 0x7f1d3c146be0 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.488+0000 7f1d4246b640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c161750 con 0x7f1d3c15e680 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.488+0000 7f1d4246b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c161a30 con 0x7f1d3c0d01d0 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.488+0000 7f1d40a08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d3c15e680 0x7f1d3c0b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d40a08640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1d3c15e680 0x7f1d3c0b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45700/0 (socket says 172.21.15.111:45700) 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d3bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1d3c146be0 0x7f1d3c160f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38428/0 (socket says 172.21.15.111:38428) 2023-12-04T05:46:32.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d40a08640 1 -- 172.21.15.111:0/2621916042 learned_addr learned my addr 172.21.15.111:0/2621916042 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d3b7fe640 1 --2- 172.21.15.111:0/2621916042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d40a08640 1 -- 172.21.15.111:0/2621916042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 msgr2=0x7f1d3c15e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d40a08640 1 --2- 172.21.15.111:0/2621916042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c15e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d40a08640 1 -- 172.21.15.111:0/2621916042 >> v1:172.21.15.111:6789/0 conn(0x7f1d3c146be0 legacy=0x7f1d3c160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d3affd640 1 -- 172.21.15.111:0/2621916042 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370287219 0 0) 0x7f1d3c15dcb0 con 0x7f1d3c146be0 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.489+0000 7f1d40a08640 1 -- 172.21.15.111:0/2621916042 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c161d10 con 0x7f1d3c15e680 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d40a08640 1 --2- 172.21.15.111:0/2621916042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c0b1f20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009e50 tx=0x7f1d3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda8c097ad5843ac server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d3affd640 1 -- 172.21.15.111:0/2621916042 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3402ca70 con 0x7f1d3c15e680 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d3affd640 1 -- 172.21.15.111:0/2621916042 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3402cbd0 con 0x7f1d3c15e680 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d3affd640 1 -- 172.21.15.111:0/2621916042 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d340343f0 con 0x7f1d3c15e680 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d4246b640 1 -- 172.21.15.111:0/2621916042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 msgr2=0x7f1d3c0b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d4246b640 1 --2- 172.21.15.111:0/2621916042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c0b1f20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009e50 tx=0x7f1d3402bed0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d4246b640 1 -- 172.21.15.111:0/2621916042 shutdown_connections 2023-12-04T05:46:32.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d4246b640 1 --2- 172.21.15.111:0/2621916042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c0b1f20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.490+0000 7f1d4246b640 1 --2- 172.21.15.111:0/2621916042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c15e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.491+0000 7f1d4246b640 1 -- 172.21.15.111:0/2621916042 >> 172.21.15.111:0/2621916042 conn(0x7f1d3c0b5500 msgr2=0x7f1d3c0b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.491+0000 7f1d4246b640 1 -- 172.21.15.111:0/2621916042 shutdown_connections 2023-12-04T05:46:32.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.491+0000 7f1d4246b640 1 -- 172.21.15.111:0/2621916042 wait complete. 2023-12-04T05:46:32.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.492+0000 7f1d4246b640 1 Processor -- start 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.492+0000 7f1d4246b640 1 -- start start 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c0c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c146be0 0x7f1d3c1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1f6220 con 0x7f1d3c15e680 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1f7010 con 0x7f1d3c146be0 2023-12-04T05:46:32.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1f6400 con 0x7f1d3c0d01d0 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c15ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45712/0 (socket says 172.21.15.111:45712) 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 -- 172.21.15.111:0/895329550 learned_addr learned my addr 172.21.15.111:0/895329550 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d3b7fe640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c146be0 0x7f1d3c1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d3bfff640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c0c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 -- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 msgr2=0x7f1d3c0c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c0c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 -- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c146be0 msgr2=0x7f1d3c1f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c146be0 0x7f1d3c1f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 -- 172.21.15.111:0/895329550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009b30 con 0x7f1d3c15e680 2023-12-04T05:46:32.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d40a08640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c15ea90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036040 tx=0x7f1d34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:46:32.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3403e450 con 0x7f1d3c15e680 2023-12-04T05:46:32.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3403e5b0 con 0x7f1d3c15e680 2023-12-04T05:46:32.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.493+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c1f66e0 con 0x7f1d3c15e680 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.494+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3403e760 con 0x7f1d3c15e680 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.494+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1f6ca0 con 0x7f1d3c15e680 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.495+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c1f6e60 con 0x7f1d3c15e680 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.495+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f1d3403e8c0 con 0x7f1d3c15e680 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.495+0000 7f1d38ff9640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1d2806b2a0 0x7f1d2806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.495+0000 7f1d3bfff640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1d2806b2a0 0x7f1d2806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.495+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1d340ac800 con 0x7f1d3c15e680 2023-12-04T05:46:32.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.496+0000 7f1d3bfff640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1d2806b2a0 0x7f1d2806d760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c000f60 tx=0x7f1d2c007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.499+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d34077590 con 0x7f1d3c15e680 2023-12-04T05:46:32.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 Processor -- start 2023-12-04T05:46:32.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 -- start start 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe66415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6640ce940 0x7fe6640cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6640cd320 con 0x7fe66415b0c0 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6640cedf0 con 0x7fe6640ce940 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe669761640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640cbee0 con 0x7fe664072250 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.603+0000 7fe6637fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6640ce940 0x7fe6640cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6637fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6640ce940 0x7fe6640cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45720/0 (socket says 172.21.15.111:45720) 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe662ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe66415b0c0 0x7fe664071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38444/0 (socket says 172.21.15.111:38444) 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6637fe640 1 -- 172.21.15.111:0/896441902 learned_addr learned my addr 172.21.15.111:0/896441902 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6627fc640 1 --2- 172.21.15.111:0/896441902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe66415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6637fe640 1 -- 172.21.15.111:0/896441902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 msgr2=0x7fe66415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6637fe640 1 --2- 172.21.15.111:0/896441902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe66415ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6637fe640 1 -- 172.21.15.111:0/896441902 >> v1:172.21.15.111:6789/0 conn(0x7fe66415b0c0 legacy=0x7fe664071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe6637fe640 1 -- 172.21.15.111:0/896441902 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6640cc710 con 0x7fe6640ce940 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.604+0000 7fe661ffb640 1 -- 172.21.15.111:0/896441902 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426452730 0 0) 0x7fe6640cd320 con 0x7fe66415b0c0 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe6637fe640 1 --2- 172.21.15.111:0/896441902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6640ce940 0x7fe6640cb9a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009e50 tx=0x7fe65c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb1325801ff6bcc server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe661ffb640 1 -- 172.21.15.111:0/896441902 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c02ca70 con 0x7fe6640ce940 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe661ffb640 1 -- 172.21.15.111:0/896441902 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c02cbd0 con 0x7fe6640ce940 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe661ffb640 1 -- 172.21.15.111:0/896441902 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0343f0 con 0x7fe6640ce940 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 -- 172.21.15.111:0/896441902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6640ce940 msgr2=0x7fe6640cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 --2- 172.21.15.111:0/896441902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6640ce940 0x7fe6640cb9a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009e50 tx=0x7fe65c02bed0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 -- 172.21.15.111:0/896441902 shutdown_connections 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 --2- 172.21.15.111:0/896441902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6640ce940 0x7fe6640cb9a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 --2- 172.21.15.111:0/896441902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe66415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.609 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 -- 172.21.15.111:0/896441902 >> 172.21.15.111:0/896441902 conn(0x7fe6640b0cf0 msgr2=0x7fe6640b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 -- 172.21.15.111:0/896441902 shutdown_connections 2023-12-04T05:46:32.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.605+0000 7fe669761640 1 -- 172.21.15.111:0/896441902 wait complete. 2023-12-04T05:46:32.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 Processor -- start 2023-12-04T05:46:32.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 -- start start 2023-12-04T05:46:32.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe6640b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.613 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6640ce940 0x7fe664159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 0x7fe664159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe664157d30 con 0x7fe66415b0c0 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640ad340 con 0x7fe6640ce940 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe669761640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640aca10 con 0x7fe664072250 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe6637fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 0x7fe664159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe6637fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 0x7fe664159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45736/0 (socket says 172.21.15.111:45736) 2023-12-04T05:46:32.614 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.607+0000 7fe6637fe640 1 -- 172.21.15.111:0/1361043318 learned_addr learned my addr 172.21.15.111:0/1361043318 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe662ffd640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe6640b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6637fe640 1 -- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 msgr2=0x7fe6640b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6637fe640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe6640b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6637fe640 1 -- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6640ce940 msgr2=0x7fe664159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6627fc640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6640ce940 0x7fe664159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6637fe640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6640ce940 0x7fe664159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6637fe640 1 -- 172.21.15.111:0/1361043318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c009b30 con 0x7fe66415b0c0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe6637fe640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 0x7fe664159c40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009e50 tx=0x7fe65c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c009510 con 0x7fe66415b0c0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c03e450 con 0x7fe66415b0c0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6640accf0 con 0x7fe66415b0c0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c03e800 con 0x7fe66415b0c0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.608+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6640ae160 con 0x7fe66415b0c0 2023-12-04T05:46:32.615 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.609+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6640ad4f0 con 0x7fe66415b0c0 2023-12-04T05:46:32.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.610+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe65c03e960 con 0x7fe66415b0c0 2023-12-04T05:46:32.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.610+0000 7fe64b7fe640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe65006b2d0 0x7fe65006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.610+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe65c0acab0 con 0x7fe66415b0c0 2023-12-04T05:46:32.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.610+0000 7fe662ffd640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe65006b2d0 0x7fe65006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.611+0000 7fe662ffd640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe65006b2d0 0x7fe65006d790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe654000f60 tx=0x7fe654009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.615+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6640ad4f0 con 0x7fe66415b0c0 2023-12-04T05:46:32.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.676+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d3c067460 con 0x7f1d3c15e680 2023-12-04T05:46:32.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.677+0000 7f1d38ff9640 1 -- 172.21.15.111:0/895329550 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d34033e80 con 0x7f1d3c15e680 2023-12-04T05:46:32.678 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1d2806b2a0 msgr2=0x7f1d2806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1d2806b2a0 0x7f1d2806d760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c000f60 tx=0x7f1d2c007c00 comp rx=0 tx=0).stop 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 msgr2=0x7f1d3c15ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c15ea90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036040 tx=0x7f1d34004a40 comp rx=0 tx=0).stop 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 shutdown_connections 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f1d2806b2a0 0x7f1d2806d760 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d3c15e680 0x7f1d3c15ea90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d3c146be0 0x7f1d3c1f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.681 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 --2- 172.21.15.111:0/895329550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d3c0d01d0 0x7f1d3c0c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 >> 172.21.15.111:0/895329550 conn(0x7f1d3c0b5500 msgr2=0x7f1d3c0b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 shutdown_connections 2023-12-04T05:46:32.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.679+0000 7f1d4246b640 1 -- 172.21.15.111:0/895329550 wait complete. 2023-12-04T05:46:32.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:46:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.790+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6640ad4b0 con 0x7fe66415b0c0 2023-12-04T05:46:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.791+0000 7fe64b7fe640 1 -- 172.21.15.111:0/1361043318 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe65c0778e0 con 0x7fe66415b0c0 2023-12-04T05:46:32.792 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:46:32.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe65006b2d0 msgr2=0x7fe65006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe65006b2d0 0x7fe65006d790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe654000f60 tx=0x7fe654009040 comp rx=0 tx=0).stop 2023-12-04T05:46:32.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 msgr2=0x7fe664159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 0x7fe664159c40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009e50 tx=0x7fe65c004a40 comp rx=0 tx=0).stop 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 shutdown_connections 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe65006b2d0 0x7fe65006d790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe66415b0c0 0x7fe664159c40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6640ce940 0x7fe664159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 --2- 172.21.15.111:0/1361043318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe664072250 0x7fe6640b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 >> 172.21.15.111:0/1361043318 conn(0x7fe6640b0cf0 msgr2=0x7fe664071930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 shutdown_connections 2023-12-04T05:46:32.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.793+0000 7fe669761640 1 -- 172.21.15.111:0/1361043318 wait complete. 2023-12-04T05:46:32.799 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.798+0000 7fd2cd2b4640 1 Processor -- start 2023-12-04T05:46:32.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.798+0000 7fd2cd2b4640 1 -- start start 2023-12-04T05:46:32.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2cd2b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b20a0 0x7fd2c80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2cd2b4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2c80b2ad0 0x7fd2c80ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2cd2b4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c80cdcb0 con 0x7fd2c80d1780 2023-12-04T05:46:32.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2cd2b4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80d39e0 con 0x7fd2c80b2ad0 2023-12-04T05:46:32.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2cd2b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80d3cc0 con 0x7fd2c80b20a0 2023-12-04T05:46:32.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2c80b2ad0 0x7fd2c80ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd2c80b2ad0 0x7fd2c80ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45738/0 (socket says 172.21.15.111:45738) 2023-12-04T05:46:32.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 -- 172.21.15.111:0/2709886968 learned_addr learned my addr 172.21.15.111:0/2709886968 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c67fc640 1 --2- 172.21.15.111:0/2709886968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b20a0 0x7fd2c80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 -- 172.21.15.111:0/2709886968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b20a0 msgr2=0x7fd2c80d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 --2- 172.21.15.111:0/2709886968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b20a0 0x7fd2c80d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 -- 172.21.15.111:0/2709886968 >> v1:172.21.15.192:6789/0 conn(0x7fd2c80d1780 legacy=0x7fd2c80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.799+0000 7fd2c77fe640 1 -- 172.21.15.111:0/2709886968 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80cdf50 con 0x7fd2c80b2ad0 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.800+0000 7fd2c77fe640 1 --2- 172.21.15.111:0/2709886968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b2ad0 0x7fd2c80ce750 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009e50 tx=0x7fd2c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5c3beff955842b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.800+0000 7fd2c5ffb640 1 -- 172.21.15.111:0/2709886968 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c002ca70 con 0x7fd2c80b2ad0 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.800+0000 7fd2c5ffb640 1 -- 172.21.15.111:0/2709886968 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c002cbd0 con 0x7fd2c80b2ad0 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.800+0000 7fd2c5ffb640 1 -- 172.21.15.111:0/2709886968 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00343f0 con 0x7fd2c80b2ad0 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/2709886968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b2ad0 msgr2=0x7fd2c80ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/2709886968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b2ad0 0x7fd2c80ce750 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009e50 tx=0x7fd2c002bed0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/2709886968 shutdown_connections 2023-12-04T05:46:32.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/2709886968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b2ad0 0x7fd2c80ce750 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/2709886968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b20a0 0x7fd2c80d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/2709886968 >> 172.21.15.111:0/2709886968 conn(0x7fd2c809c4e0 msgr2=0x7fd2c80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/2709886968 shutdown_connections 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.801+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/2709886968 wait complete. 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.802+0000 7fd2cd2b4640 1 Processor -- start 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.802+0000 7fd2cd2b4640 1 -- start start 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2cd2b4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 0x7fd2c8151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2cd2b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b2ad0 0x7fd2c80b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2cd2b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2c80d1780 0x7fd2c815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2cd2b4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80c85e0 con 0x7fd2c80b20a0 2023-12-04T05:46:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2cd2b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80ca270 con 0x7fd2c80d1780 2023-12-04T05:46:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2cd2b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8160820 con 0x7fd2c80b2ad0 2023-12-04T05:46:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2c6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 0x7fd2c8151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2c6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 0x7fd2c8151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45742/0 (socket says 172.21.15.111:45742) 2023-12-04T05:46:32.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.803+0000 7fd2c6ffd640 1 -- 172.21.15.111:0/1675380423 learned_addr learned my addr 172.21.15.111:0/1675380423 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:46:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c77fe640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2c80d1780 0x7fd2c815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c67fc640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b2ad0 0x7fd2c80b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c6ffd640 1 -- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b2ad0 msgr2=0x7fd2c80b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c6ffd640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b2ad0 0x7fd2c80b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c6ffd640 1 -- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2c80d1780 msgr2=0x7fd2c815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c6ffd640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2c80d1780 0x7fd2c815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c6ffd640 1 -- 172.21.15.111:0/1675380423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c0009b30 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c77fe640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2c80d1780 0x7fd2c815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2c6ffd640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 0x7fd2c8151790 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c400 tx=0x7fd2bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0052d0 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc005430 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c80c7680 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc003830 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.804+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c80c7c40 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.805+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c80c8720 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.805+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd2bc003990 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.806+0000 7fd2af7fe640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd2b406b2a0 0x7fd2b406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.806+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd2bc0891b0 con 0x7fd2c80b20a0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.806+0000 7fd2c67fc640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd2b406b2a0 0x7fd2b406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.806+0000 7fd2c67fc640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd2b406b2a0 0x7fd2b406d760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80b1ca0 tx=0x7fd2b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.810+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2c80c8720 con 0x7fd2c80b20a0 2023-12-04T05:46:32.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.912+0000 7f084de7c640 1 Processor -- start 2023-12-04T05:46:32.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.913+0000 7f084de7c640 1 -- start start 2023-12-04T05:46:32.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f084de7c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0848144ba0 0x7f08480ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f084de7c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f08480b2630 0x7f084815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f084de7c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08480cd480 con 0x7f08480ce970 2023-12-04T05:46:32.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f084de7c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084815b750 con 0x7f0848144ba0 2023-12-04T05:46:32.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f084de7c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084815fd70 con 0x7f08480b2630 2023-12-04T05:46:32.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0848144ba0 0x7f08480ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0848144ba0 0x7f08480ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45746/0 (socket says 172.21.15.111:45746) 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f0846ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f08480ce970 0x7f08480b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38456/0 (socket says 172.21.15.111:38456) 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 -- 172.21.15.111:0/2825513574 learned_addr learned my addr 172.21.15.111:0/2825513574 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f0847fff640 1 --2- 172.21.15.111:0/2825513574 >> v2:172.21.15.192:3300/0 conn(0x7f08480b2630 0x7f084815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 -- 172.21.15.111:0/2825513574 >> v2:172.21.15.192:3300/0 conn(0x7f08480b2630 msgr2=0x7f084815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 --2- 172.21.15.111:0/2825513574 >> v2:172.21.15.192:3300/0 conn(0x7f08480b2630 0x7f084815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 -- 172.21.15.111:0/2825513574 >> v1:172.21.15.111:6789/0 conn(0x7f08480ce970 legacy=0x7f08480b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08477fe640 1 -- 172.21.15.111:0/2825513574 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084815f160 con 0x7f0848144ba0 2023-12-04T05:46:32.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.914+0000 7f08467fc640 1 -- 172.21.15.111:0/2825513574 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268274391 0 0) 0x7f0838001110 con 0x7f08480ce970 2023-12-04T05:46:32.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.915+0000 7f08477fe640 1 --2- 172.21.15.111:0/2825513574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0848144ba0 0x7f08480ccf40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f08480b6240 tx=0x7f083c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d47f5f9a9e16cd server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.915+0000 7f08467fc640 1 -- 172.21.15.111:0/2825513574 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c030860 con 0x7f0848144ba0 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.915+0000 7f08467fc640 1 -- 172.21.15.111:0/2825513574 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c0309c0 con 0x7f0848144ba0 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.915+0000 7f08467fc640 1 -- 172.21.15.111:0/2825513574 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c030d50 con 0x7f0848144ba0 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 -- 172.21.15.111:0/2825513574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0848144ba0 msgr2=0x7f08480ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 --2- 172.21.15.111:0/2825513574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0848144ba0 0x7f08480ccf40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f08480b6240 tx=0x7f083c02b9a0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 -- 172.21.15.111:0/2825513574 shutdown_connections 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 --2- 172.21.15.111:0/2825513574 >> v2:172.21.15.192:3300/0 conn(0x7f08480b2630 0x7f084815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 --2- 172.21.15.111:0/2825513574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0848144ba0 0x7f08480ccf40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 -- 172.21.15.111:0/2825513574 >> 172.21.15.111:0/2825513574 conn(0x7f08480b5500 msgr2=0x7f08480b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 -- 172.21.15.111:0/2825513574 shutdown_connections 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.916+0000 7f084de7c640 1 -- 172.21.15.111:0/2825513574 wait complete. 2023-12-04T05:46:32.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.917+0000 7f084de7c640 1 Processor -- start 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.917+0000 7f084de7c640 1 -- start start 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f084de7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 0x7f0848143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f084de7c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480ce970 0x7f08481f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f084de7c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0848144ba0 0x7f08480b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f084de7c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480ad8e0 con 0x7f08480b2630 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f084de7c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480adbc0 con 0x7f0848144ba0 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f084de7c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480adea0 con 0x7f08480ce970 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f08477fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 0x7f0848143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f08477fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 0x7f0848143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45758/0 (socket says 172.21.15.111:45758) 2023-12-04T05:46:32.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.918+0000 7f08477fe640 1 -- 172.21.15.111:0/390614161 learned_addr learned my addr 172.21.15.111:0/390614161 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f08477fe640 1 -- 172.21.15.111:0/390614161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480ce970 msgr2=0x7f08481f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f08477fe640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480ce970 0x7f08481f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f08477fe640 1 -- 172.21.15.111:0/390614161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0848144ba0 msgr2=0x7f08480b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f08477fe640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0848144ba0 0x7f08480b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f08477fe640 1 -- 172.21.15.111:0/390614161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c0094d0 con 0x7f08480b2630 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f08477fe640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 0x7f0848143a70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f083c000c00 tx=0x7f083c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c039a10 con 0x7f08480b2630 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c039b70 con 0x7f08480b2630 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08480ae180 con 0x7f08480b2630 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c043410 con 0x7f08480b2630 2023-12-04T05:46:32.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.919+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0848143fb0 con 0x7f08480b2630 2023-12-04T05:46:32.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.920+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08480ae6a0 con 0x7f08480b2630 2023-12-04T05:46:32.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.920+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f083c02f3b0 con 0x7f08480b2630 2023-12-04T05:46:32.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.921+0000 7f082ffff640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f083406b2a0 0x7f083406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:32.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.921+0000 7f0846ffd640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f083406b2a0 0x7f083406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:32.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.921+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f083c0abdd0 con 0x7f08480b2630 2023-12-04T05:46:32.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.921+0000 7f0846ffd640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f083406b2a0 0x7f083406d760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0838007fd0 tx=0x7f0838001310 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:32.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.925+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f083c076b60 con 0x7f08480b2630 2023-12-04T05:46:32.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.985+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2c80c8720 con 0x7fd2c80b20a0 2023-12-04T05:46:32.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.985+0000 7fd2af7fe640 1 -- 172.21.15.111:0/1675380423 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd2c80c8720 con 0x7fd2c80b20a0 2023-12-04T05:46:32.989 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:46:32.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd2b406b2a0 msgr2=0x7fd2b406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd2b406b2a0 0x7fd2b406d760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80b1ca0 tx=0x7fd2b8008040 comp rx=0 tx=0).stop 2023-12-04T05:46:32.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 msgr2=0x7fd2c8151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 0x7fd2c8151790 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c400 tx=0x7fd2bc00c8c0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 shutdown_connections 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd2b406b2a0 0x7fd2b406d760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2c80d1780 0x7fd2c815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2c80b2ad0 0x7fd2c80b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 --2- 172.21.15.111:0/1675380423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c80b20a0 0x7fd2c8151790 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.987+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 >> 172.21.15.111:0/1675380423 conn(0x7fd2c809c4e0 msgr2=0x7fd2c806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.988+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 shutdown_connections 2023-12-04T05:46:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:32.988+0000 7fd2cd2b4640 1 -- 172.21.15.111:0/1675380423 wait complete. 2023-12-04T05:46:33.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:46:33.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.097+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08480ccbd0 con 0x7f08480b2630 2023-12-04T05:46:33.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.098+0000 7f082ffff640 1 -- 172.21.15.111:0/390614161 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f083c07a180 con 0x7f08480b2630 2023-12-04T05:46:33.099 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:46:33.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.099+0000 7fe2b8256640 1 Processor -- start 2023-12-04T05:46:33.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.099+0000 7fe2b8256640 1 -- start start 2023-12-04T05:46:33.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f083406b2a0 msgr2=0x7f083406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f083406b2a0 0x7f083406d760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0838007fd0 tx=0x7f0838001310 comp rx=0 tx=0).stop 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 msgr2=0x7f0848143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 0x7f0848143a70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f083c000c00 tx=0x7f083c004730 comp rx=0 tx=0).stop 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 shutdown_connections 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f083406b2a0 0x7f083406d760 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0848144ba0 0x7f08480b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08480ce970 0x7f08481f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 --2- 172.21.15.111:0/390614161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08480b2630 0x7f0848143a70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 >> 172.21.15.111:0/390614161 conn(0x7f08480b5500 msgr2=0x7f0848071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 shutdown_connections 2023-12-04T05:46:33.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7f084de7c640 1 -- 172.21.15.111:0/390614161 wait complete. 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b8256640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe2b00d1780 0x7fe2b00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b8256640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00cdcb0 con 0x7fe2b00b2ad0 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b8256640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00d39e0 con 0x7fe2b00b20a0 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b8256640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b00d3cc0 con 0x7fe2b00d1780 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b67f3640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe2b00b2ad0 0x7fe2b00ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38464/0 (socket says 172.21.15.111:38464) 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3709745656 learned_addr learned my addr 172.21.15.111:0/3709745656 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b5ff2640 1 --2- 172.21.15.111:0/3709745656 >> v2:172.21.15.192:3300/0 conn(0x7fe2b00d1780 0x7fe2b00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783358892 0 0) 0x7fe2b00cdcb0 con 0x7fe2b00b2ad0 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.100+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29c003520 con 0x7fe2b00b2ad0 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618707515 0 0) 0x7fe2b00d39e0 con 0x7fe2b00b20a0 2023-12-04T05:46:33.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b00cdcb0 con 0x7fe2b00b20a0 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 167941113 0 0) 0x7fe29c003520 con 0x7fe2b00b2ad0 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 >> v2:172.21.15.192:3300/0 conn(0x7fe2b00d1780 msgr2=0x7fe2b00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 --2- 172.21.15.111:0/3709745656 >> v2:172.21.15.192:3300/0 conn(0x7fe2b00d1780 0x7fe2b00d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 >> v1:172.21.15.192:6789/0 conn(0x7fe2b00b20a0 legacy=0x7fe2b00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b00cdf50 con 0x7fe2b00b2ad0 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fe2a80019f0 con 0x7fe2b00b2ad0 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b5ff2640 1 --2- 172.21.15.111:0/3709745656 >> v2:172.21.15.192:3300/0 conn(0x7fe2b00d1780 0x7fe2b00d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 >> v1:172.21.15.111:6789/0 conn(0x7fe2b00b2ad0 legacy=0x7fe2b00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe29c003b50 0x7fe29c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0047c0 0x7fe29c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00d3cc0 con 0x7fe29c0051d0 2023-12-04T05:46:33.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c00c100 con 0x7fe29c003b50 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c00c2c0 con 0x7fe29c0047c0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b5ff2640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe29c003b50 0x7fe29c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b57f1640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0047c0 0x7fe29c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0047c0 msgr2=0x7fe29c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0047c0 0x7fe29c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe29c003b50 msgr2=0x7fe29c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe29c003b50 0x7fe29c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3709745656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c003520 con 0x7fe29c0051d0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.101+0000 7fe2b57f1640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0047c0 0x7fe29c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.102+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3709745656 reap_dead start 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.102+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b980 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800a700 tx=0x7fe29c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4748b92251c3398b server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.102+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8034630 con 0x7fe29c0051d0 2023-12-04T05:46:33.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.102+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a802f070 con 0x7fe29c0051d0 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.102+0000 7fe2b4ff0640 1 -- 172.21.15.111:0/3709745656 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80349d0 con 0x7fe29c0051d0 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 -- 172.21.15.111:0/3709745656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe29c0051d0 msgr2=0x7fe29c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b980 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800a700 tx=0x7fe29c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 -- 172.21.15.111:0/3709745656 shutdown_connections 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3709745656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b980 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 -- 172.21.15.111:0/3709745656 >> 172.21.15.111:0/3709745656 conn(0x7fe2b009c4e0 msgr2=0x7fe2b00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 -- 172.21.15.111:0/3709745656 shutdown_connections 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.103+0000 7fe2b8256640 1 -- 172.21.15.111:0/3709745656 wait complete. 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.104+0000 7fe2b8256640 1 Processor -- start 2023-12-04T05:46:33.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.104+0000 7fe2b8256640 1 -- start start 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b8256640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0051d0 0x7fe2b00cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b8256640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b01f63a0 0x7fe2b0157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b8256640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 0x7fe2b00b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b8256640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00cb840 con 0x7fe2b00b0560 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b8256640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0157cc0 con 0x7fe2b01f63a0 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b8256640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00b0f20 con 0x7fe29c0051d0 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 0x7fe2b00b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 0x7fe2b00b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45778/0 (socket says 172.21.15.111:45778) 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3379752176 learned_addr learned my addr 172.21.15.111:0/3379752176 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b57f1640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b01f63a0 0x7fe2b0157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0051d0 msgr2=0x7fe2b00cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b5ff2640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0051d0 0x7fe2b00cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0051d0 0x7fe2b00cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b01f63a0 msgr2=0x7fe2b0157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b01f63a0 0x7fe2b0157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 -- 172.21.15.111:0/3379752176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b01f6b10 con 0x7fe2b00b0560 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b57f1640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b01f63a0 0x7fe2b0157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.105+0000 7fe2b67f3640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 0x7fe2b00b09e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8002230 tx=0x7fe2a8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.106+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8034430 con 0x7fe2b00b0560 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.106+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8034590 con 0x7fe2b00b0560 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.106+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b01f6df0 con 0x7fe2b00b0560 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.106+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8034900 con 0x7fe2b00b0560 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.106+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b00c7e40 con 0x7fe2b00b0560 2023-12-04T05:46:33.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.107+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b01f7310 con 0x7fe2b00b0560 2023-12-04T05:46:33.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.107+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fe2a8031030 con 0x7fe2b00b0560 2023-12-04T05:46:33.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.108+0000 7fe2a67fc640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe29406b2d0 0x7fe29406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.108+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe2a80ab790 con 0x7fe2b00b0560 2023-12-04T05:46:33.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.108+0000 7fe2b5ff2640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe29406b2d0 0x7fe29406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.108+0000 7fe2b5ff2640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe29406b2d0 0x7fe29406d790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009af0 tx=0x7fe2a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:46:33.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.112+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2a8076520 con 0x7fe2b00b0560 2023-12-04T05:46:33.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.217+0000 7fa0fda44640 1 Processor -- start 2023-12-04T05:46:33.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.217+0000 7fa0fda44640 1 -- start start 2023-12-04T05:46:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0fda44640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f813cee0 0x7fa0f806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0fda44640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0f815aba0 0x7fa0f806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0fda44640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f80aea20 con 0x7fa0f8140200 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0fda44640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f813d2c0 con 0x7fa0f815aba0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0fda44640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f815b3a0 con 0x7fa0f813cee0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0f815aba0 0x7fa0f806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0f815aba0 0x7fa0f806c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45794/0 (socket says 172.21.15.111:45794) 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 -- 172.21.15.111:0/2115680732 learned_addr learned my addr 172.21.15.111:0/2115680732 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f6ffd640 1 --2- 172.21.15.111:0/2115680732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f813cee0 0x7fa0f806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 -- 172.21.15.111:0/2115680732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f813cee0 msgr2=0x7fa0f806c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 --2- 172.21.15.111:0/2115680732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f813cee0 0x7fa0f806c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 -- 172.21.15.111:0/2115680732 >> v1:172.21.15.192:6789/0 conn(0x7fa0f8140200 legacy=0x7fa0f815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.218+0000 7fa0f77fe640 1 -- 172.21.15.111:0/2115680732 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f815b0c0 con 0x7fa0f815aba0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.219+0000 7fa0f77fe640 1 --2- 172.21.15.111:0/2115680732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f815aba0 0x7fa0f806c790 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009e50 tx=0x7fa0f002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e13499ef056448 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.219+0000 7fa0f5ffb640 1 -- 172.21.15.111:0/2115680732 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f002c9e0 con 0x7fa0f815aba0 2023-12-04T05:46:33.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.219+0000 7fa0f5ffb640 1 -- 172.21.15.111:0/2115680732 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f002cb40 con 0x7fa0f815aba0 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.219+0000 7fa0f5ffb640 1 -- 172.21.15.111:0/2115680732 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00343f0 con 0x7fa0f815aba0 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 -- 172.21.15.111:0/2115680732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f815aba0 msgr2=0x7fa0f806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 --2- 172.21.15.111:0/2115680732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f815aba0 0x7fa0f806c790 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009e50 tx=0x7fa0f002be40 comp rx=0 tx=0).stop 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 -- 172.21.15.111:0/2115680732 shutdown_connections 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 --2- 172.21.15.111:0/2115680732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f815aba0 0x7fa0f806c790 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 --2- 172.21.15.111:0/2115680732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f813cee0 0x7fa0f806c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 -- 172.21.15.111:0/2115680732 >> 172.21.15.111:0/2115680732 conn(0x7fa0f806a4c0 msgr2=0x7fa0f806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 -- 172.21.15.111:0/2115680732 shutdown_connections 2023-12-04T05:46:33.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.220+0000 7fa0fda44640 1 -- 172.21.15.111:0/2115680732 wait complete. 2023-12-04T05:46:33.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.221+0000 7fa0fda44640 1 Processor -- start 2023-12-04T05:46:33.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.221+0000 7fa0fda44640 1 -- start start 2023-12-04T05:46:33.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0fda44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 0x7fa0f8159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0fda44640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f8140200 0x7fa0f813ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0fda44640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f815aba0 0x7fa0f8071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0fda44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8070590 con 0x7fa0f813cee0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0fda44640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80715f0 con 0x7fa0f8140200 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0fda44640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8070830 con 0x7fa0f815aba0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 0x7fa0f8159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 0x7fa0f8159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45798/0 (socket says 172.21.15.111:45798) 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 -- 172.21.15.111:0/3454507746 learned_addr learned my addr 172.21.15.111:0/3454507746 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f77fe640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f815aba0 0x7fa0f8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f67fc640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f8140200 0x7fa0f813ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 -- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f815aba0 msgr2=0x7fa0f8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f815aba0 0x7fa0f8071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 -- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f8140200 msgr2=0x7fa0f813ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f8140200 0x7fa0f813ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 -- 172.21.15.111:0/3454507746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f0009b30 con 0x7fa0f813cee0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.222+0000 7fa0f6ffd640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 0x7fa0f8159630 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800d800 tx=0x7fa0e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.223+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800b6c0 con 0x7fa0f813cee0 2023-12-04T05:46:33.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.223+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e800b820 con 0x7fa0f813cee0 2023-12-04T05:46:33.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.223+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f8070b10 con 0x7fa0f813cee0 2023-12-04T05:46:33.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.223+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800bbb0 con 0x7fa0f813cee0 2023-12-04T05:46:33.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.223+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f80710d0 con 0x7fa0f813cee0 2023-12-04T05:46:33.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.224+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f80ae6d0 con 0x7fa0f813cee0 2023-12-04T05:46:33.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.224+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa0e800bd10 con 0x7fa0f813cee0 2023-12-04T05:46:33.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.224+0000 7fa0dffff640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa0e406b2a0 0x7fa0e406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.224+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa0e808a100 con 0x7fa0f813cee0 2023-12-04T05:46:33.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.225+0000 7fa0f67fc640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa0e406b2a0 0x7fa0e406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.228 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.225+0000 7fa0f67fc640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa0e406b2a0 0x7fa0e406d760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa0f813e150 tx=0x7fa0ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.229+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0e8054f30 con 0x7fa0f813cee0 2023-12-04T05:46:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.301+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe2b01f7310 con 0x7fe2b00b0560 2023-12-04T05:46:33.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.301+0000 7fe2a67fc640 1 -- 172.21.15.111:0/3379752176 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v49) v1 ==== 80+0+244 (secure 0 0 0) 0x7fe2a8079b40 con 0x7fe2b00b0560 2023-12-04T05:46:33.303 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6835/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:33.303 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6833/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:33.303 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6836/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:33.303 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6834/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:33.304 INFO:teuthology.orchestra.run.smithi111.stderr:listed 4 entries 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe29406b2d0 msgr2=0x7fe29406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe29406b2d0 0x7fe29406d790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009af0 tx=0x7fe2a0008040 comp rx=0 tx=0).stop 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 msgr2=0x7fe2b00b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 0x7fe2b00b09e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8002230 tx=0x7fe2a8001cb0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 shutdown_connections 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fe29406b2d0 0x7fe29406d790 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b00b0560 0x7fe2b00b09e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b01f63a0 0x7fe2b0157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 --2- 172.21.15.111:0/3379752176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe29c0051d0 0x7fe2b00cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 >> 172.21.15.111:0/3379752176 conn(0x7fe2b009c4e0 msgr2=0x7fe2b00b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 shutdown_connections 2023-12-04T05:46:33.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.303+0000 7fe2b8256640 1 -- 172.21.15.111:0/3379752176 wait complete. 2023-12-04T05:46:33.315 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:46:33.316 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:33.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:33.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.419+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0f80693d0 con 0x7fa0f813cee0 2023-12-04T05:46:33.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.419+0000 7fa0dffff640 1 -- 172.21.15.111:0/3454507746 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v49) v1 ==== 80+0+244 (secure 0 0 0) 0x7fa0e8058550 con 0x7fa0f813cee0 2023-12-04T05:46:33.421 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6835/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:33.422 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6833/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:33.422 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6836/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:33.422 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6834/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:33.422 INFO:teuthology.orchestra.run.smithi111.stderr:listed 4 entries 2023-12-04T05:46:33.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.421+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa0e406b2a0 msgr2=0x7fa0e406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.421+0000 7fa0fda44640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa0e406b2a0 0x7fa0e406d760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa0f813e150 tx=0x7fa0ec009040 comp rx=0 tx=0).stop 2023-12-04T05:46:33.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.421+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 msgr2=0x7fa0f8159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.421+0000 7fa0fda44640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 0x7fa0f8159630 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800d800 tx=0x7fa0e800dcc0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 shutdown_connections 2023-12-04T05:46:33.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa0e406b2a0 0x7fa0e406d760 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f815aba0 0x7fa0f8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f8140200 0x7fa0f813ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 --2- 172.21.15.111:0/3454507746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f813cee0 0x7fa0f8159630 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 >> 172.21.15.111:0/3454507746 conn(0x7fa0f806a4c0 msgr2=0x7fa0f80ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 shutdown_connections 2023-12-04T05:46:33.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.422+0000 7fa0fda44640 1 -- 172.21.15.111:0/3454507746 wait complete. 2023-12-04T05:46:33.434 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-12-04T05:46:33.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:33.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-04T05:46:33.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:46:33.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:33.461 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-04T05:46:33.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.471+0000 7fc36b9be640 1 Processor -- start 2023-12-04T05:46:33.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.472+0000 7fc36b9be640 1 -- start start 2023-12-04T05:46:33.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.472+0000 7fc36b9be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc36415e2e0 0x7fc3641681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.472+0000 7fc36b9be640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc364160e40 con 0x7fc3640aee90 2023-12-04T05:46:33.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.472+0000 7fc36b9be640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc364161100 con 0x7fc3640ad6b0 2023-12-04T05:46:33.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.472+0000 7fc36b9be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3641613e0 con 0x7fc36415e2e0 2023-12-04T05:46:33.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc36975a640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc3640aee90 0x7fc3640af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38472/0 (socket says 172.21.15.111:38472) 2023-12-04T05:46:33.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc36975a640 1 -- 172.21.15.111:0/1975312180 learned_addr learned my addr 172.21.15.111:0/1975312180 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:33.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc369f5b640 1 --2- 172.21.15.111:0/1975312180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc36415e2e0 0x7fc3641681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc35bfff640 1 -- 172.21.15.111:0/1975312180 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680972402 0 0) 0x7fc364160e40 con 0x7fc3640aee90 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc35bfff640 1 -- 172.21.15.111:0/1975312180 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc350003520 con 0x7fc3640aee90 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc369f5b640 1 -- 172.21.15.111:0/1975312180 >> v1:172.21.15.192:6789/0 conn(0x7fc3640ad6b0 legacy=0x7fc36415dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc35bfff640 1 -- 172.21.15.111:0/1975312180 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943214027 0 0) 0x7fc364161100 con 0x7fc3640ad6b0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc369f5b640 1 -- 172.21.15.111:0/1975312180 >> v1:172.21.15.111:6789/0 conn(0x7fc3640aee90 legacy=0x7fc3640af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.473+0000 7fc369f5b640 1 -- 172.21.15.111:0/1975312180 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3641616c0 con 0x7fc36415e2e0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.474+0000 7fc369f5b640 1 --2- 172.21.15.111:0/1975312180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc36415e2e0 0x7fc3641681d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc360009dc0 tx=0x7fc36002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cf98fd7f4cab3f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.474+0000 7fc35bfff640 1 -- 172.21.15.111:0/1975312180 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36002ccd0 con 0x7fc36415e2e0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.474+0000 7fc35bfff640 1 -- 172.21.15.111:0/1975312180 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36002ce30 con 0x7fc36415e2e0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.474+0000 7fc35bfff640 1 -- 172.21.15.111:0/1975312180 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360034640 con 0x7fc36415e2e0 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 -- 172.21.15.111:0/1975312180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc36415e2e0 msgr2=0x7fc3641681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1975312180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc36415e2e0 0x7fc3641681d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc360009dc0 tx=0x7fc36002bf20 comp rx=0 tx=0).stop 2023-12-04T05:46:33.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 -- 172.21.15.111:0/1975312180 shutdown_connections 2023-12-04T05:46:33.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1975312180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc36415e2e0 0x7fc3641681d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 -- 172.21.15.111:0/1975312180 >> 172.21.15.111:0/1975312180 conn(0x7fc3640b1e10 msgr2=0x7fc3640b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 -- 172.21.15.111:0/1975312180 shutdown_connections 2023-12-04T05:46:33.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.475+0000 7fc36b9be640 1 -- 172.21.15.111:0/1975312180 wait complete. 2023-12-04T05:46:33.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.476+0000 7fc36b9be640 1 Processor -- start 2023-12-04T05:46:33.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.476+0000 7fc36b9be640 1 -- start start 2023-12-04T05:46:33.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.476+0000 7fc36b9be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3640ad6b0 0x7fc364146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.476+0000 7fc36b9be640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 0x7fc36415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.476+0000 7fc36b9be640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc36415e2e0 0x7fc36414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc36b9be640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364155f20 con 0x7fc3640aee90 2023-12-04T05:46:33.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc36b9be640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406ab90 con 0x7fc36415e2e0 2023-12-04T05:46:33.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc36b9be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36406ae70 con 0x7fc3640ad6b0 2023-12-04T05:46:33.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 0x7fc36415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 0x7fc36415fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45812/0 (socket says 172.21.15.111:45812) 2023-12-04T05:46:33.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 -- 172.21.15.111:0/1489200087 learned_addr learned my addr 172.21.15.111:0/1489200087 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc36975a640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3640ad6b0 0x7fc364146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc369f5b640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc36415e2e0 0x7fc36414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 -- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3640ad6b0 msgr2=0x7fc364146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3640ad6b0 0x7fc364146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 -- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc36415e2e0 msgr2=0x7fc36414d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc36415e2e0 0x7fc36414d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 -- 172.21.15.111:0/1489200087 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360009a70 con 0x7fc3640aee90 2023-12-04T05:46:33.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc368f59640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 0x7fc36415fc40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c520 tx=0x7fc35400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3540035f0 con 0x7fc3640aee90 2023-12-04T05:46:33.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc354003770 con 0x7fc3640aee90 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354003b20 con 0x7fc3640aee90 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.477+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36406b150 con 0x7fc3640aee90 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.478+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc364151350 con 0x7fc3640aee90 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.479+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364067460 con 0x7fc3640aee90 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.479+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc354003c80 con 0x7fc3640aee90 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.479+0000 7fc359ffb640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34806b2a0 0x7fc34806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.480+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fc354089210 con 0x7fc3640aee90 2023-12-04T05:46:33.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.480+0000 7fc36975a640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34806b2a0 0x7fc34806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.480+0000 7fc36975a640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34806b2a0 0x7fc34806d760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005880 tx=0x7fc35c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.485+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc354054050 con 0x7fc3640aee90 2023-12-04T05:46:33.488 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:46:33.488 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:33.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:33.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.627+0000 7fb3d65a9640 1 Processor -- start 2023-12-04T05:46:33.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d65a9640 1 -- start start 2023-12-04T05:46:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d65a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d65a9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3d00b20a0 0x7fb3d00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d65a9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d00cdcb0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d65a9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00d20a0 con 0x7fb3d00b20a0 2023-12-04T05:46:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d65a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00d39e0 con 0x7fb3d00d1780 2023-12-04T05:46:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3cf7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3d00b20a0 0x7fb3d00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3cf7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3d00b20a0 0x7fb3d00d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45822/0 (socket says 172.21.15.111:45822) 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1425660431 learned_addr learned my addr 172.21.15.111:0/1425660431 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.628+0000 7fb3d4b46640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb3d00b2ad0 0x7fb3d00ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38484/0 (socket says 172.21.15.111:38484) 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cffff640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1425660431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 msgr2=0x7fb3d00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cf7fe640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1425660431 >> v1:172.21.15.111:6789/0 conn(0x7fb3d00b2ad0 legacy=0x7fb3d00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1425660431 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00d3cc0 con 0x7fb3d00b20a0 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3ceffd640 1 -- 172.21.15.111:0/1425660431 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634554480 0 0) 0x7fb3d00cdcb0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cffff640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.629+0000 7fb3cf7fe640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b20a0 0x7fb3d00d1b60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009870 tx=0x7fb3c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239527b11c6db3f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3ceffd640 1 -- 172.21.15.111:0/1425660431 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0031640 con 0x7fb3d00b20a0 2023-12-04T05:46:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3ceffd640 1 -- 172.21.15.111:0/1425660431 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c00317a0 con 0x7fb3d00b20a0 2023-12-04T05:46:33.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3ceffd640 1 -- 172.21.15.111:0/1425660431 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0031b30 con 0x7fb3d00b20a0 2023-12-04T05:46:33.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1425660431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b20a0 msgr2=0x7fb3d00d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b20a0 0x7fb3d00d1b60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009870 tx=0x7fb3c002b8f0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1425660431 shutdown_connections 2023-12-04T05:46:33.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b20a0 0x7fb3d00d1b60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1425660431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1425660431 >> 172.21.15.111:0/1425660431 conn(0x7fb3d009c4e0 msgr2=0x7fb3d00d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1425660431 shutdown_connections 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.630+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1425660431 wait complete. 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.631+0000 7fb3d65a9640 1 Processor -- start 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 -- start start 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d00b20a0 0x7fb3d0157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 0x7fb3d00ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00cb0c0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c7150 con 0x7fb3d00b20a0 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3d65a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00cade0 con 0x7fb3d00d1780 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 0x7fb3d00ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 0x7fb3d00ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45834/0 (socket says 172.21.15.111:45834) 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.632+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1038513963 learned_addr learned my addr 172.21.15.111:0/1038513963 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3d4b46640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cffff640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d00b20a0 0x7fb3d0157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 msgr2=0x7fb3d00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cf7fe640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d00b20a0 msgr2=0x7fb3d0157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cf7fe640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d00b20a0 0x7fb3d0157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cf7fe640 1 -- 172.21.15.111:0/1038513963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c00094d0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3cf7fe640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 0x7fb3d00ca270 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000c00 tx=0x7fb3c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c003ac60 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c003ade0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d00c6b00 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0044630 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.633+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d015ceb0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.634+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d00c6960 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.635+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fb3c002f050 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.635+0000 7fb3ccff9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3bc06b2a0 0x7fb3bc06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.635+0000 7fb3cffff640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3bc06b2a0 0x7fb3bc06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.635+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb3c00abbb0 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.636+0000 7fb3cffff640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3bc06b2a0 0x7fb3bc06d760 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ab60 tx=0x7fb3c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.639+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3d00c6960 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.662+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3640b2bd0 con 0x7fc3640aee90 2023-12-04T05:46:33.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.663+0000 7fc359ffb640 1 -- 172.21.15.111:0/1489200087 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7fc354057670 con 0x7fc3640aee90 2023-12-04T05:46:33.665 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:33.665 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:33.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34806b2a0 msgr2=0x7fc34806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34806b2a0 0x7fc34806d760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005880 tx=0x7fc35c009040 comp rx=0 tx=0).stop 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 msgr2=0x7fc36415fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 0x7fc36415fc40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c520 tx=0x7fc35400c9e0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 shutdown_connections 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc34806b2a0 0x7fc34806d760 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc36415e2e0 0x7fc36414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3640aee90 0x7fc36415fc40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 --2- 172.21.15.111:0/1489200087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3640ad6b0 0x7fc364146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 >> 172.21.15.111:0/1489200087 conn(0x7fc3640b1e10 msgr2=0x7fc3640b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 shutdown_connections 2023-12-04T05:46:33.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.665+0000 7fc36b9be640 1 -- 172.21.15.111:0/1489200087 wait complete. 2023-12-04T05:46:33.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:41:00.843728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:41:01.612301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:46:33.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:46:33.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.781+0000 7f5596631640 1 Processor -- start 2023-12-04T05:46:33.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.781+0000 7f5596631640 1 -- start start 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5596631640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f559015d480 0x7f55900d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5596631640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55900d1310 0x7f55900cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5596631640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f559015d860 0x7f55900d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5596631640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559015ddc0 con 0x7f559015d860 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5596631640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559015e0a0 con 0x7f559015d480 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5596631640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559015e380 con 0x7f55900d1310 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f559015d860 0x7f55900d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f559015d860 0x7f55900d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45836/0 (socket says 172.21.15.111:45836) 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 -- 172.21.15.111:0/1877219697 learned_addr learned my addr 172.21.15.111:0/1877219697 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f558ffff640 1 --2- 172.21.15.111:0/1877219697 >> v2:172.21.15.192:3300/0 conn(0x7f559015d480 0x7f55900d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f558f7fe640 1 --2- 172.21.15.111:0/1877219697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55900d1310 0x7f55900cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 -- 172.21.15.111:0/1877219697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55900d1310 msgr2=0x7f55900cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 --2- 172.21.15.111:0/1877219697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55900d1310 0x7f55900cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 -- 172.21.15.111:0/1877219697 >> v2:172.21.15.192:3300/0 conn(0x7f559015d480 msgr2=0x7f55900d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 --2- 172.21.15.111:0/1877219697 >> v2:172.21.15.192:3300/0 conn(0x7f559015d480 0x7f55900d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f5594bce640 1 -- 172.21.15.111:0/1877219697 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559015e660 con 0x7f559015d860 2023-12-04T05:46:33.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.782+0000 7f558ffff640 1 --2- 172.21.15.111:0/1877219697 >> v2:172.21.15.192:3300/0 conn(0x7f559015d480 0x7f55900d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:33.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.783+0000 7f5594bce640 1 --2- 172.21.15.111:0/1877219697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559015d860 0x7f55900d0010 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f55900b5d30 tx=0x7f558802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cefd46457049af3 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f558effd640 1 -- 172.21.15.111:0/1877219697 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558802ca70 con 0x7f559015d860 2023-12-04T05:46:33.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f558effd640 1 -- 172.21.15.111:0/1877219697 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558802cbd0 con 0x7f559015d860 2023-12-04T05:46:33.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f558effd640 1 -- 172.21.15.111:0/1877219697 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55880343f0 con 0x7f559015d860 2023-12-04T05:46:33.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 -- 172.21.15.111:0/1877219697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559015d860 msgr2=0x7f55900d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 --2- 172.21.15.111:0/1877219697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559015d860 0x7f55900d0010 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f55900b5d30 tx=0x7f558802bed0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 -- 172.21.15.111:0/1877219697 shutdown_connections 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 --2- 172.21.15.111:0/1877219697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559015d860 0x7f55900d0010 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 --2- 172.21.15.111:0/1877219697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55900d1310 0x7f55900cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 --2- 172.21.15.111:0/1877219697 >> v2:172.21.15.192:3300/0 conn(0x7f559015d480 0x7f55900d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 -- 172.21.15.111:0/1877219697 >> 172.21.15.111:0/1877219697 conn(0x7f559009c540 msgr2=0x7f55900d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 -- 172.21.15.111:0/1877219697 shutdown_connections 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.784+0000 7f5596631640 1 -- 172.21.15.111:0/1877219697 wait complete. 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.785+0000 7f5596631640 1 Processor -- start 2023-12-04T05:46:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 -- start start 2023-12-04T05:46:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 0x7f55900b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559015d480 0x7f55900cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559015d860 0x7f559014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55900b61c0 con 0x7f55900d1310 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55900cb150 con 0x7f559015d480 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f5596631640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55900cb430 con 0x7f559015d860 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f558ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 0x7f55900b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f558ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 0x7f55900b89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45844/0 (socket says 172.21.15.111:45844) 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.786+0000 7f558ffff640 1 -- 172.21.15.111:0/1488872740 learned_addr learned my addr 172.21.15.111:0/1488872740 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f5594bce640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559015d860 0x7f559014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558f7fe640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559015d480 0x7f55900cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558ffff640 1 -- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559015d860 msgr2=0x7f559014e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558ffff640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559015d860 0x7f559014e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558ffff640 1 -- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559015d480 msgr2=0x7f55900cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558ffff640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559015d480 0x7f55900cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558ffff640 1 -- 172.21.15.111:0/1488872740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588009b30 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f5594bce640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559015d860 0x7f559014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558ffff640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 0x7f55900b89a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f558400c7f0 tx=0x7f558400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584007a30 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584007b90 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55900cb710 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558400ce40 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.787+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55901f35a0 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.788+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5590067980 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.788+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f55840113f0 con 0x7f55900d1310 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.789+0000 7f558cff9640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f557c06b2a0 0x7f557c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.789+0000 7f558f7fe640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f557c06b2a0 0x7f557c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.789+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5584089e20 con 0x7f55900d1310 2023-12-04T05:46:33.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.789+0000 7f558f7fe640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f557c06b2a0 0x7f557c06d760 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f55900b8920 tx=0x7f5580008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.793+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5584054c30 con 0x7f55900d1310 2023-12-04T05:46:33.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.817+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3d00c6960 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.818+0000 7fb3ccff9640 1 -- 172.21.15.111:0/1038513963 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7fb3d00c6960 con 0x7fb3d00b2ad0 2023-12-04T05:46:33.820 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:33.820 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.820+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3bc06b2a0 msgr2=0x7fb3bc06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.820+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3bc06b2a0 0x7fb3bc06d760 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400ab60 tx=0x7fb3c4008040 comp rx=0 tx=0).stop 2023-12-04T05:46:33.822 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 msgr2=0x7fb3d00ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 0x7fb3d00ca270 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000c00 tx=0x7fb3c00046c0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 shutdown_connections 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fb3bc06b2a0 0x7fb3bc06d760 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d00d1780 0x7fb3d00ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3d00b2ad0 0x7fb3d00ca270 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 --2- 172.21.15.111:0/1038513963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d00b20a0 0x7fb3d0157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 >> 172.21.15.111:0/1038513963 conn(0x7fb3d009c4e0 msgr2=0x7fb3d00b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 shutdown_connections 2023-12-04T05:46:33.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.821+0000 7fb3d65a9640 1 -- 172.21.15.111:0/1038513963 wait complete. 2023-12-04T05:46:33.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:41:00.843728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:41:01.612301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:46:33.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:46:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 Processor -- start 2023-12-04T05:46:33.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 -- start start 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbc4006b740 0x7fbc4015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc4013eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc400ccab0 con 0x7fbc4013f060 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4013f880 con 0x7fbc4006b740 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.937+0000 7fbc46e61640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4013fb60 con 0x7fbc4006bb20 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc453fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbc4013f060 0x7fbc400b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38492/0 (socket says 172.21.15.111:38492) 2023-12-04T05:46:33.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc453fe640 1 -- 172.21.15.111:0/1178081583 learned_addr learned my addr 172.21.15.111:0/1178081583 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/1178081583 >> v2:172.21.15.192:3300/0 conn(0x7fbc4006b740 0x7fbc4015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc3ffff640 1 --2- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc4013eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc3f7fe640 1 -- 172.21.15.111:0/1178081583 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110548111 0 0) 0x7fbc400ccab0 con 0x7fbc4013f060 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc3f7fe640 1 -- 172.21.15.111:0/1178081583 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2c003520 con 0x7fbc4013f060 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc44bfd640 1 -- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 msgr2=0x7fbc4013eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc4013eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc44bfd640 1 -- 172.21.15.111:0/1178081583 >> v1:172.21.15.111:6789/0 conn(0x7fbc4013f060 legacy=0x7fbc400b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc3f7fe640 1 -- 172.21.15.111:0/1178081583 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 545157232 0 0) 0x7fbc2c003520 con 0x7fbc4013f060 2023-12-04T05:46:33.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.938+0000 7fbc44bfd640 1 -- 172.21.15.111:0/1178081583 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4013fe40 con 0x7fbc4006b740 2023-12-04T05:46:33.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.939+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015e2b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009d90 tx=0x7fbc3802bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e021cf80171112f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.939+0000 7fbc3f7fe640 1 -- 172.21.15.111:0/1178081583 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28001670 con 0x7fbc4006b740 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.939+0000 7fbc3f7fe640 1 -- 172.21.15.111:0/1178081583 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3802c8a0 con 0x7fbc4006b740 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.939+0000 7fbc3f7fe640 1 -- 172.21.15.111:0/1178081583 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3802cc10 con 0x7fbc4006b740 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 -- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 msgr2=0x7fbc4015e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 --2- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015e2b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009d90 tx=0x7fbc3802bd70 comp rx=0 tx=0).stop 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 -- 172.21.15.111:0/1178081583 shutdown_connections 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 --2- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc4013eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 --2- 172.21.15.111:0/1178081583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015e2b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 -- 172.21.15.111:0/1178081583 >> 172.21.15.111:0/1178081583 conn(0x7fbc40071030 msgr2=0x7fbc40071440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 -- 172.21.15.111:0/1178081583 shutdown_connections 2023-12-04T05:46:33.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.940+0000 7fbc46e61640 1 -- 172.21.15.111:0/1178081583 wait complete. 2023-12-04T05:46:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.942+0000 7fbc46e61640 1 Processor -- start 2023-12-04T05:46:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.942+0000 7fbc46e61640 1 -- start start 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc46e61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc46e61640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc400cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc46e61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 0x7fbc4015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc46e61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4015cc10 con 0x7fbc4013f060 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc46e61640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4015ab00 con 0x7fbc4006b740 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc46e61640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4015c270 con 0x7fbc4006bb20 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.943+0000 7fbc453fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 0x7fbc4015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 0x7fbc4015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45860/0 (socket says 172.21.15.111:45860) 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 -- 172.21.15.111:0/3400624411 learned_addr learned my addr 172.21.15.111:0/3400624411 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc3ffff640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc400cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 -- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 msgr2=0x7fbc400cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc400cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 -- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 msgr2=0x7fbc4015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 -- 172.21.15.111:0/3400624411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38009a70 con 0x7fbc4013f060 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc453fe640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 0x7fbc4015a5c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc280017a0 tx=0x7fbc2800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28005ad0 con 0x7fbc4013f060 2023-12-04T05:46:33.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc28005c30 con 0x7fbc4013f060 2023-12-04T05:46:33.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28007070 con 0x7fbc4013f060 2023-12-04T05:46:33.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4015c550 con 0x7fbc4013f060 2023-12-04T05:46:33.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.944+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4015bf20 con 0x7fbc4013f060 2023-12-04T05:46:33.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.945+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4015cd50 con 0x7fbc4013f060 2023-12-04T05:46:33.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.946+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbc280071d0 con 0x7fbc4013f060 2023-12-04T05:46:33.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.946+0000 7fbc3d7fa640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc2006b2a0 0x7fbc2006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:33.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.946+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc2006b2a0 0x7fbc2006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:33.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.946+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fbc280898b0 con 0x7fbc4013f060 2023-12-04T05:46:33.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.947+0000 7fbc44bfd640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc2006b2a0 0x7fbc2006d760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbc38036000 tx=0x7fbc38030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:33.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.950+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc4015cd50 con 0x7fbc4013f060 2023-12-04T05:46:33.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.970+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55900b6300 con 0x7f55900d1310 2023-12-04T05:46:33.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.970+0000 7f558cff9640 1 -- 172.21.15.111:0/1488872740 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55900b6300 con 0x7f55900d1310 2023-12-04T05:46:33.975 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:46:33.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f557c06b2a0 msgr2=0x7f557c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f557c06b2a0 0x7f557c06d760 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f55900b8920 tx=0x7f5580008040 comp rx=0 tx=0).stop 2023-12-04T05:46:33.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 msgr2=0x7f55900b89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 0x7f55900b89a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f558400c7f0 tx=0x7f558400cbf0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 shutdown_connections 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f557c06b2a0 0x7f557c06d760 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559015d860 0x7f559014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f559015d480 0x7f55900cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 --2- 172.21.15.111:0/1488872740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55900d1310 0x7f55900b89a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 >> 172.21.15.111:0/1488872740 conn(0x7f559009c540 msgr2=0x7f55901611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 shutdown_connections 2023-12-04T05:46:33.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:33.972+0000 7f5596631640 1 -- 172.21.15.111:0/1488872740 wait complete. 2023-12-04T05:46:33.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:46:34.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.086+0000 7f826331d640 1 Processor -- start 2023-12-04T05:46:34.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.087+0000 7f826331d640 1 -- start start 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.087+0000 7f826331d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 0x7f825c0d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.087+0000 7f826331d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f825c0d1310 0x7f825c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.087+0000 7f826331d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c15ddc0 con 0x7f825c15d860 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.087+0000 7f826331d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c15e0a0 con 0x7f825c0d1310 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.087+0000 7f826331d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c15e380 con 0x7f825c15d480 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82610b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 0x7f825c0d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82608b8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f825c0d1310 0x7f825c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82618ba640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f825c15d860 0x7f825c160270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50468/0 (socket says 172.21.15.111:50468) 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82618ba640 1 -- 172.21.15.111:0/2028014471 learned_addr learned my addr 172.21.15.111:0/2028014471 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82610b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 0x7f825c0d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:43220/0 (socket says 172.21.15.111:43220) 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82610b9640 1 -- 172.21.15.111:0/2028014471 >> v2:172.21.15.192:3300/0 conn(0x7f825c0d1310 msgr2=0x7f825c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82610b9640 1 --2- 172.21.15.111:0/2028014471 >> v2:172.21.15.192:3300/0 conn(0x7f825c0d1310 0x7f825c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82610b9640 1 -- 172.21.15.111:0/2028014471 >> v1:172.21.15.192:6789/0 conn(0x7f825c15d860 legacy=0x7f825c160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f8253fff640 1 -- 172.21.15.111:0/2028014471 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091593379 0 0) 0x7f825c15ddc0 con 0x7f825c15d860 2023-12-04T05:46:34.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.088+0000 7f82610b9640 1 -- 172.21.15.111:0/2028014471 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c15e660 con 0x7f825c15d480 2023-12-04T05:46:34.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.089+0000 7f82610b9640 1 --2- 172.21.15.111:0/2028014471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 0x7f825c0d0dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f824c0097a0 tx=0x7f824c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9ac7bbecf44792 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.089+0000 7f8253fff640 1 -- 172.21.15.111:0/2028014471 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c031830 con 0x7f825c15d480 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.089+0000 7f8253fff640 1 -- 172.21.15.111:0/2028014471 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c031990 con 0x7f825c15d480 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.089+0000 7f8253fff640 1 -- 172.21.15.111:0/2028014471 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c031d20 con 0x7f825c15d480 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.089+0000 7f826331d640 1 -- 172.21.15.111:0/2028014471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 msgr2=0x7f825c0d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.089+0000 7f826331d640 1 --2- 172.21.15.111:0/2028014471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 0x7f825c0d0dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f824c0097a0 tx=0x7f824c02b8d0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.090+0000 7f826331d640 1 -- 172.21.15.111:0/2028014471 shutdown_connections 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.090+0000 7f826331d640 1 --2- 172.21.15.111:0/2028014471 >> v2:172.21.15.192:3300/0 conn(0x7f825c0d1310 0x7f825c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.090+0000 7f826331d640 1 --2- 172.21.15.111:0/2028014471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c15d480 0x7f825c0d0dd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.090+0000 7f826331d640 1 -- 172.21.15.111:0/2028014471 >> 172.21.15.111:0/2028014471 conn(0x7f825c09c540 msgr2=0x7f825c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.090+0000 7f826331d640 1 -- 172.21.15.111:0/2028014471 shutdown_connections 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.090+0000 7f826331d640 1 -- 172.21.15.111:0/2028014471 wait complete. 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 Processor -- start 2023-12-04T05:46:34.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 -- start start 2023-12-04T05:46:34.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c0d1310 0x7f825c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f825c15d480 0x7f825c0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 0x7f825c0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cb380 con 0x7f825c15d860 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c14c500 con 0x7f825c15d480 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.091+0000 7f826331d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cb5d0 con 0x7f825c0d1310 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 0x7f825c0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 0x7f825c0c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45866/0 (socket says 172.21.15.111:45866) 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 -- 172.21.15.111:0/3241259142 learned_addr learned my addr 172.21.15.111:0/3241259142 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82610b9640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c0d1310 0x7f825c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82608b8640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f825c15d480 0x7f825c0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 -- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c0d1310 msgr2=0x7f825c14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c0d1310 0x7f825c14bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 -- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f825c15d480 msgr2=0x7f825c0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f825c15d480 0x7f825c0c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 -- 172.21.15.111:0/3241259142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c009400 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f82618ba640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 0x7f825c0c7660 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f825400c6d0 tx=0x7f825400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254004d60 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400a640 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c0cb8b0 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.092+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400aa70 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.093+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c0cbe70 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.094+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c067980 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.094+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f82540070d0 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.094+0000 7f8251ffb640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f824006b2a0 0x7f824006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.095+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f82540894f0 con 0x7f825c15d860 2023-12-04T05:46:34.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.095+0000 7f82610b9640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f824006b2a0 0x7f824006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.095+0000 7f82610b9640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f824006b2a0 0x7f824006d760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f824c036000 tx=0x7f824c03e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.099+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82540543b0 con 0x7f825c15d860 2023-12-04T05:46:34.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.127+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc4015cd50 con 0x7fbc4013f060 2023-12-04T05:46:34.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.128+0000 7fbc3d7fa640 1 -- 172.21.15.111:0/3400624411 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc4015cd50 con 0x7fbc4013f060 2023-12-04T05:46:34.130 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc2006b2a0 msgr2=0x7fbc2006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc2006b2a0 0x7fbc2006d760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbc38036000 tx=0x7fbc38030000 comp rx=0 tx=0).stop 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 msgr2=0x7fbc4015a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 0x7fbc4015a5c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc280017a0 tx=0x7fbc2800cbe0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 shutdown_connections 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbc2006b2a0 0x7fbc2006d760 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc4013f060 0x7fbc4015a5c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc4006bb20 0x7fbc400cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 --2- 172.21.15.111:0/3400624411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc4006b740 0x7fbc4015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 >> 172.21.15.111:0/3400624411 conn(0x7fbc40071030 msgr2=0x7fbc4015e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 shutdown_connections 2023-12-04T05:46:34.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.130+0000 7fbc46e61640 1 -- 172.21.15.111:0/3400624411 wait complete. 2023-12-04T05:46:34.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:46:34.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.245+0000 7fa9a9c51640 1 Processor -- start 2023-12-04T05:46:34.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.245+0000 7fa9a9c51640 1 -- start start 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a9c51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a415d480 0x7fa9a40d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a9c51640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa9a40d01e0 0x7fa9a4160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a9c51640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a415ddc0 con 0x7fa9a40d1310 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a9c51640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a415e0a0 con 0x7fa9a40d01e0 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a9c51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a415e380 con 0x7fa9a415d480 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a2ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa9a40d1310 0x7fa9a40cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38498/0 (socket says 172.21.15.111:38498) 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a2ffd640 1 -- 172.21.15.111:0/1190749872 learned_addr learned my addr 172.21.15.111:0/1190749872 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a3fff640 1 --2- 172.21.15.111:0/1190749872 >> v2:172.21.15.111:3300/0 conn(0x7fa9a40d01e0 0x7fa9a4160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a27fc640 1 -- 172.21.15.111:0/1190749872 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306273805 0 0) 0x7fa9a415ddc0 con 0x7fa9a40d1310 2023-12-04T05:46:34.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a37fe640 1 --2- 172.21.15.111:0/1190749872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a415d480 0x7fa9a40d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a27fc640 1 -- 172.21.15.111:0/1190749872 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa98c003520 con 0x7fa9a40d1310 2023-12-04T05:46:34.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a3fff640 1 -- 172.21.15.111:0/1190749872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a415d480 msgr2=0x7fa9a40d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a3fff640 1 --2- 172.21.15.111:0/1190749872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a415d480 0x7fa9a40d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a3fff640 1 -- 172.21.15.111:0/1190749872 >> v1:172.21.15.111:6789/0 conn(0x7fa9a40d1310 legacy=0x7fa9a40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.246+0000 7fa9a3fff640 1 -- 172.21.15.111:0/1190749872 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a415e660 con 0x7fa9a40d01e0 2023-12-04T05:46:34.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.247+0000 7fa9a3fff640 1 --2- 172.21.15.111:0/1190749872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a40d01e0 0x7fa9a4160270 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009d90 tx=0x7fa99c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c66a9331e6b164 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.247+0000 7fa9a27fc640 1 -- 172.21.15.111:0/1190749872 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c02cc50 con 0x7fa9a40d01e0 2023-12-04T05:46:34.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.247+0000 7fa9a27fc640 1 -- 172.21.15.111:0/1190749872 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c02cdb0 con 0x7fa9a40d01e0 2023-12-04T05:46:34.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.247+0000 7fa9a27fc640 1 -- 172.21.15.111:0/1190749872 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c034660 con 0x7fa9a40d01e0 2023-12-04T05:46:34.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 -- 172.21.15.111:0/1190749872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a40d01e0 msgr2=0x7fa9a4160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/1190749872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a40d01e0 0x7fa9a4160270 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009d90 tx=0x7fa99c02bea0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 -- 172.21.15.111:0/1190749872 shutdown_connections 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/1190749872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a40d01e0 0x7fa9a4160270 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/1190749872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a415d480 0x7fa9a40d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 -- 172.21.15.111:0/1190749872 >> 172.21.15.111:0/1190749872 conn(0x7fa9a409c540 msgr2=0x7fa9a40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 -- 172.21.15.111:0/1190749872 shutdown_connections 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.248+0000 7fa9a9c51640 1 -- 172.21.15.111:0/1190749872 wait complete. 2023-12-04T05:46:34.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.249+0000 7fa9a9c51640 1 Processor -- start 2023-12-04T05:46:34.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.249+0000 7fa9a9c51640 1 -- start start 2023-12-04T05:46:34.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a9c51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a40d01e0 0x7fa9a415c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a9c51640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a40d1310 0x7fa9a40b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a9c51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 0x7fa9a40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a9c51640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a415cd50 con 0x7fa9a415d480 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a9c51640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40b01a0 con 0x7fa9a40d1310 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a9c51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40b0480 con 0x7fa9a40d01e0 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 0x7fa9a40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 0x7fa9a40cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45896/0 (socket says 172.21.15.111:45896) 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 -- 172.21.15.111:0/2475238276 learned_addr learned my addr 172.21.15.111:0/2475238276 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a2ffd640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a40d1310 0x7fa9a40b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a37fe640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a40d01e0 0x7fa9a415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 -- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a40d01e0 msgr2=0x7fa9a415c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a40d01e0 0x7fa9a415c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 -- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a40d1310 msgr2=0x7fa9a40b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a40d1310 0x7fa9a40b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a3fff640 1 -- 172.21.15.111:0/2475238276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009a70 con 0x7fa9a415d480 2023-12-04T05:46:34.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.250+0000 7fa9a2ffd640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a40d1310 0x7fa9a40b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.251+0000 7fa9a3fff640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 0x7fa9a40cc990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009d90 tx=0x7fa99c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.251+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c040650 con 0x7fa9a415d480 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.251+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c0407b0 con 0x7fa9a415d480 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.251+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a40b0760 con 0x7fa9a415d480 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.251+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c040b60 con 0x7fa9a415d480 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.251+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a40b0d20 con 0x7fa9a415d480 2023-12-04T05:46:34.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.252+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a4067960 con 0x7fa9a415d480 2023-12-04T05:46:34.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.253+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fa99c040cc0 con 0x7fa9a415d480 2023-12-04T05:46:34.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.253+0000 7fa98bfff640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa99006b2a0 0x7fa99006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.253+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa99c0ac9f0 con 0x7fa9a415d480 2023-12-04T05:46:34.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.253+0000 7fa9a37fe640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa99006b2a0 0x7fa99006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.254+0000 7fa9a37fe640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa99006b2a0 0x7fa99006d760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa994006fd0 tx=0x7fa994008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.258+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa99c077800 con 0x7fa9a415d480 2023-12-04T05:46:34.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.275+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f825c0cc090 con 0x7f825c15d860 2023-12-04T05:46:34.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.275+0000 7f8251ffb640 1 -- 172.21.15.111:0/3241259142 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82540579d0 con 0x7f825c15d860 2023-12-04T05:46:34.277 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:46:34.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f824006b2a0 msgr2=0x7f824006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f824006b2a0 0x7f824006d760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f824c036000 tx=0x7f824c03e040 comp rx=0 tx=0).stop 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 msgr2=0x7f825c0c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 0x7f825c0c7660 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f825400c6d0 tx=0x7f825400cb90 comp rx=0 tx=0).stop 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 shutdown_connections 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f824006b2a0 0x7f824006d760 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f825c15d860 0x7f825c0c7660 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f825c15d480 0x7f825c0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 --2- 172.21.15.111:0/3241259142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f825c0d1310 0x7f825c14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 >> 172.21.15.111:0/3241259142 conn(0x7f825c09c540 msgr2=0x7f825c160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 shutdown_connections 2023-12-04T05:46:34.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.277+0000 7f826331d640 1 -- 172.21.15.111:0/3241259142 wait complete. 2023-12-04T05:46:34.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:46:34.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.392+0000 7f7868209640 1 Processor -- start 2023-12-04T05:46:34.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.392+0000 7f7868209640 1 -- start start 2023-12-04T05:46:34.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.393+0000 7f7868209640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f786015d860 0x7f78600cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.393+0000 7f7868209640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78600d0010 0x7f7860160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.393+0000 7f7868209640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786015ddc0 con 0x7f786015d480 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.393+0000 7f7868209640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78600d04c0 con 0x7f786015d860 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.393+0000 7f7868209640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786015e0a0 con 0x7f78600d0010 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78657a4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f786015d860 0x7f78600cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78667a6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78600d0010 0x7f7860160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f7865fa5640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f786015d480 0x7f78600d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50472/0 (socket says 172.21.15.111:50472) 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78657a4640 1 --2- 172.21.15.111:0/64961235 >> v2:172.21.15.192:3300/0 conn(0x7f786015d860 0x7f78600cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:37760/0 (socket says 172.21.15.111:37760) 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f7865fa5640 1 -- 172.21.15.111:0/64961235 learned_addr learned my addr 172.21.15.111:0/64961235 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78667a6640 1 -- 172.21.15.111:0/64961235 >> v2:172.21.15.192:3300/0 conn(0x7f786015d860 msgr2=0x7f78600cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78667a6640 1 --2- 172.21.15.111:0/64961235 >> v2:172.21.15.192:3300/0 conn(0x7f786015d860 0x7f78600cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78667a6640 1 -- 172.21.15.111:0/64961235 >> v1:172.21.15.192:6789/0 conn(0x7f786015d480 legacy=0x7f78600d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.394+0000 7f78667a6640 1 -- 172.21.15.111:0/64961235 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786015e380 con 0x7f78600d0010 2023-12-04T05:46:34.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f78667a6640 1 --2- 172.21.15.111:0/64961235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78600d0010 0x7f7860160270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7858009dc0 tx=0x7f785802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d110e0f812675a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7864fa3640 1 -- 172.21.15.111:0/64961235 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785802cc40 con 0x7f78600d0010 2023-12-04T05:46:34.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7864fa3640 1 -- 172.21.15.111:0/64961235 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785802cda0 con 0x7f78600d0010 2023-12-04T05:46:34.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7864fa3640 1 -- 172.21.15.111:0/64961235 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858034660 con 0x7f78600d0010 2023-12-04T05:46:34.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 -- 172.21.15.111:0/64961235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78600d0010 msgr2=0x7f7860160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 --2- 172.21.15.111:0/64961235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78600d0010 0x7f7860160270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7858009dc0 tx=0x7f785802be90 comp rx=0 tx=0).stop 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 -- 172.21.15.111:0/64961235 shutdown_connections 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 --2- 172.21.15.111:0/64961235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78600d0010 0x7f7860160270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 --2- 172.21.15.111:0/64961235 >> v2:172.21.15.192:3300/0 conn(0x7f786015d860 0x7f78600cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 -- 172.21.15.111:0/64961235 >> 172.21.15.111:0/64961235 conn(0x7f786009c540 msgr2=0x7f78600d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 -- 172.21.15.111:0/64961235 shutdown_connections 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.395+0000 7f7868209640 1 -- 172.21.15.111:0/64961235 wait complete. 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.396+0000 7f7868209640 1 Processor -- start 2023-12-04T05:46:34.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 -- start start 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 0x7f78601560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f786015d480 0x7f78601540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f786015d860 0x7f78600cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600b8aa0 con 0x7f78600d0010 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600b87c0 con 0x7f786015d480 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7868209640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78601e7f60 con 0x7f786015d860 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7865fa5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 0x7f78601560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7865fa5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 0x7f78601560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45904/0 (socket says 172.21.15.111:45904) 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f7865fa5640 1 -- 172.21.15.111:0/1964792030 learned_addr learned my addr 172.21.15.111:0/1964792030 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f78657a4640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f786015d480 0x7f78601540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.397+0000 7f78667a6640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f786015d860 0x7f78600cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7865fa5640 1 -- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f786015d860 msgr2=0x7f78600cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7865fa5640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f786015d860 0x7f78600cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7865fa5640 1 -- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f786015d480 msgr2=0x7f78601540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7865fa5640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f786015d480 0x7f78601540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7865fa5640 1 -- 172.21.15.111:0/1964792030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009a70 con 0x7f78600d0010 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7865fa5640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 0x7f78601560d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f785000c4a0 tx=0x7f785000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850005ce0 con 0x7f78600d0010 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7850005e40 con 0x7f78600d0010 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850007270 con 0x7f78600d0010 2023-12-04T05:46:34.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78601e8240 con 0x7f78600d0010 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.398+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78600ccf00 con 0x7f78600d0010 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.399+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860067980 con 0x7f78600d0010 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.400+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f7850007490 con 0x7f78600d0010 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.400+0000 7f78567fc640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f784406b2a0 0x7f784406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.400+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7850089100 con 0x7f78600d0010 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.400+0000 7f78657a4640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f784406b2a0 0x7f784406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.401+0000 7f78657a4640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f784406b2a0 0x7f784406d760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f78600b6a50 tx=0x7f785c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.406 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.404+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7850053f30 con 0x7f78600d0010 2023-12-04T05:46:34.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.432+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9a40b0f10 con 0x7fa9a415d480 2023-12-04T05:46:34.436 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.433+0000 7fa98bfff640 1 -- 172.21.15.111:0/2475238276 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa99c009450 con 0x7fa9a415d480 2023-12-04T05:46:34.436 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:46:34.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa99006b2a0 msgr2=0x7fa99006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa99006b2a0 0x7fa99006d760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa994006fd0 tx=0x7fa994008040 comp rx=0 tx=0).stop 2023-12-04T05:46:34.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 msgr2=0x7fa9a40cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 0x7fa9a40cc990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009d90 tx=0x7fa99c004a40 comp rx=0 tx=0).stop 2023-12-04T05:46:34.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 shutdown_connections 2023-12-04T05:46:34.437 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fa99006b2a0 0x7fa99006d760 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9a415d480 0x7fa9a40cc990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a40d1310 0x7fa9a40b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 --2- 172.21.15.111:0/2475238276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9a40d01e0 0x7fa9a415c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 >> 172.21.15.111:0/2475238276 conn(0x7fa9a409c540 msgr2=0x7fa9a40b7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 shutdown_connections 2023-12-04T05:46:34.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.435+0000 7fa9a9c51640 1 -- 172.21.15.111:0/2475238276 wait complete. 2023-12-04T05:46:34.447 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:46:34.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.550+0000 7f12b8489640 1 Processor -- start 2023-12-04T05:46:34.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.550+0000 7f12b8489640 1 -- start start 2023-12-04T05:46:34.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.551+0000 7f12b8489640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f12b01660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.551+0000 7f12b8489640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b0144e60 con 0x7f12b0144a80 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.551+0000 7f12b8489640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b015f050 con 0x7f12b00cd2e0 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.551+0000 7f12b8489640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b015f330 con 0x7f12b00b2710 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.551+0000 7f12b6225640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f12b0144a80 0x7f12b00ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38510/0 (socket says 172.21.15.111:38510) 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.551+0000 7f12b6225640 1 -- 172.21.15.111:0/1697206616 learned_addr learned my addr 172.21.15.111:0/1697206616 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b6a26640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f12b01660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522704632 0 0) 0x7f12a4001110 con 0x7f12b0144a80 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f129c003520 con 0x7f12b0144a80 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 273032045 0 0) 0x7f129c003520 con 0x7f12b0144a80 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 msgr2=0x7f12b01660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f12b01660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 >> v1:172.21.15.192:6789/0 conn(0x7f12b00cd2e0 legacy=0x7f12b00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b015f610 con 0x7f12b0144a80 2023-12-04T05:46:34.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f12a4001820 con 0x7f12b0144a80 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 >> v1:172.21.15.111:6789/0 conn(0x7f12b0144a80 legacy=0x7f12b00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f129c0039a0 0x7f129c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f129c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f129c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b015f330 con 0x7f129c0047c0 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c00ba60 con 0x7f129c0039a0 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c00bc20 con 0x7f12b00b2710 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5a24640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f129c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b6225640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f129c0039a0 0x7f129c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b6a26640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f129c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5a24640 1 -- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 msgr2=0x7f129c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5a24640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f129c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5a24640 1 -- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f129c0039a0 msgr2=0x7f129c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5a24640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f129c0039a0 0x7f129c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b5a24640 1 -- 172.21.15.111:0/1697206616 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c003520 con 0x7f129c0047c0 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.552+0000 7f12b6a26640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b00b2710 0x7f129c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.553+0000 7f12b5a24640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f129c004c40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f12ac001750 tx=0x7f129c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e66c8aaa269599 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.553+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0052d0 con 0x7f129c0047c0 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.553+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac02e050 con 0x7f129c0047c0 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.553+0000 7f12b5223640 1 -- 172.21.15.111:0/1697206616 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0035f0 con 0x7f129c0047c0 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b8489640 1 -- 172.21.15.111:0/1697206616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 msgr2=0x7f129c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b8489640 1 --2- 172.21.15.111:0/1697206616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f129c004c40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f12ac001750 tx=0x7f129c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b6a26640 1 -- 172.21.15.111:0/1697206616 reap_dead start 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b8489640 1 -- 172.21.15.111:0/1697206616 shutdown_connections 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b8489640 1 -- 172.21.15.111:0/1697206616 >> 172.21.15.111:0/1697206616 conn(0x7f12b00b5500 msgr2=0x7f12b00b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b8489640 1 -- 172.21.15.111:0/1697206616 shutdown_connections 2023-12-04T05:46:34.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.554+0000 7f12b8489640 1 -- 172.21.15.111:0/1697206616 wait complete. 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.555+0000 7f12b8489640 1 Processor -- start 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.555+0000 7f12b8489640 1 -- start start 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f12b013a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b0154f10 0x7f12b01e7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b01e4bb0 0x7f12b01e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b013adb0 con 0x7f129c0047c0 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b015d1f0 con 0x7f12b01e4bb0 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b015c5e0 con 0x7f12b0154f10 2023-12-04T05:46:34.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f12b013a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f12b013a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45914/0 (socket says 172.21.15.111:45914) 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 -- 172.21.15.111:0/2854158647 learned_addr learned my addr 172.21.15.111:0/2854158647 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b5a24640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b0154f10 0x7f12b01e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6a26640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b01e4bb0 0x7f12b01e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 -- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b0154f10 msgr2=0x7f12b01e7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b0154f10 0x7f12b01e7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 -- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b01e4bb0 msgr2=0x7f12b01e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b01e4bb0 0x7f12b01e5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 -- 172.21.15.111:0/2854158647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b015c8c0 con 0x7f129c0047c0 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6a26640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b01e4bb0 0x7f12b01e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b5a24640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b0154f10 0x7f12b01e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b6225640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f12b013a870 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f12a4001340 tx=0x7f12a40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4007740 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a400f940 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b015cba0 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.556+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a400fd10 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.557+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b01e7860 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.558+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0067460 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.558+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f12a401d460 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.558+0000 7f12a2ffd640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f129406b2d0 0x7f129406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.558+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f12a4089c20 con 0x7f129c0047c0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.558+0000 7f12b5a24640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f129406b2d0 0x7f129406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.559+0000 7f12b5a24640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f129406b2d0 0x7f129406d790 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f12ac033000 tx=0x7f12ac009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.562+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12a4058430 con 0x7f129c0047c0 2023-12-04T05:46:34.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.582+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7860069360 con 0x7f78600d0010 2023-12-04T05:46:34.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.582+0000 7f78567fc640 1 -- 172.21.15.111:0/1964792030 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7850057550 con 0x7f78600d0010 2023-12-04T05:46:34.584 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f784406b2a0 msgr2=0x7f784406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f784406b2a0 0x7f784406d760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f78600b6a50 tx=0x7f785c009190 comp rx=0 tx=0).stop 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 msgr2=0x7f78601560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 0x7f78601560d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f785000c4a0 tx=0x7f785000c960 comp rx=0 tx=0).stop 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 shutdown_connections 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f784406b2a0 0x7f784406d760 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f786015d860 0x7f78600cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f786015d480 0x7f78601540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 --2- 172.21.15.111:0/1964792030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78600d0010 0x7f78601560d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 >> 172.21.15.111:0/1964792030 conn(0x7f786009c540 msgr2=0x7f786015f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 shutdown_connections 2023-12-04T05:46:34.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.584+0000 7f7868209640 1 -- 172.21.15.111:0/1964792030 wait complete. 2023-12-04T05:46:34.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:46:34.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.697+0000 7f6dc2e36640 1 Processor -- start 2023-12-04T05:46:34.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.697+0000 7f6dc2e36640 1 -- start start 2023-12-04T05:46:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.698+0000 7f6dc2e36640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6dbc06e7e0 0x7f6dbc06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.698+0000 7f6dc2e36640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.698+0000 7f6dc2e36640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc06f100 con 0x7f6dbc15ed10 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.698+0000 7f6dc2e36640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc15fc30 con 0x7f6dbc06e7e0 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.698+0000 7f6dc2e36640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc13e380 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc13d3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc0bd2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6dbc06e7e0 0x7f6dbc06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc13d3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:43306/0 (socket says 172.21.15.111:43306) 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc13d3640 1 -- 172.21.15.111:0/3350040005 learned_addr learned my addr 172.21.15.111:0/3350040005 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc0bd2640 1 -- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 msgr2=0x7f6dbc0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc0bd2640 1 --2- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc0bd2640 1 -- 172.21.15.111:0/3350040005 >> v1:172.21.15.192:6789/0 conn(0x7f6dbc15ed10 legacy=0x7f6dbc15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc0bd2640 1 -- 172.21.15.111:0/3350040005 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc15b810 con 0x7f6dbc06e7e0 2023-12-04T05:46:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.699+0000 7f6dc13d3640 1 --2- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc0bd2640 1 --2- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc06ebc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6db00097a0 tx=0x7f6db002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53dcfb3a5272b6a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dbb7fe640 1 -- 172.21.15.111:0/3350040005 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db002e070 con 0x7f6dbc06e7e0 2023-12-04T05:46:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dbb7fe640 1 -- 172.21.15.111:0/3350040005 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00315a0 con 0x7f6dbc06e7e0 2023-12-04T05:46:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dbb7fe640 1 -- 172.21.15.111:0/3350040005 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0031930 con 0x7f6dbc06e7e0 2023-12-04T05:46:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 -- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 msgr2=0x7f6dbc06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc06ebc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6db00097a0 tx=0x7f6db002b840 comp rx=0 tx=0).stop 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 -- 172.21.15.111:0/3350040005 shutdown_connections 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/3350040005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc06ebc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 -- 172.21.15.111:0/3350040005 >> 172.21.15.111:0/3350040005 conn(0x7f6dbc06a8b0 msgr2=0x7f6dbc06acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 -- 172.21.15.111:0/3350040005 shutdown_connections 2023-12-04T05:46:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.700+0000 7f6dc2e36640 1 -- 172.21.15.111:0/3350040005 wait complete. 2023-12-04T05:46:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 Processor -- start 2023-12-04T05:46:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 -- start start 2023-12-04T05:46:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc15ed10 0x7f6dbc1396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc155d30 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc155f10 con 0x7f6dbc06e7e0 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.702+0000 7f6dc2e36640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc1561f0 con 0x7f6dbc15ed10 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45918/0 (socket says 172.21.15.111:45918) 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 -- 172.21.15.111:0/643027789 learned_addr learned my addr 172.21.15.111:0/643027789 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dc13d3640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc15ed10 0x7f6dbc1396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dc0bd2640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 -- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc15ed10 msgr2=0x7f6dbc1396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc15ed10 0x7f6dbc1396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 -- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 msgr2=0x7f6dbc158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 -- 172.21.15.111:0/643027789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4009b30 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dbbfff640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0b1650 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c400 tx=0x7f6da800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80052d0 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8005430 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db0009400 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8003830 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.703+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc1567b0 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.704+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc159060 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.705+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f6da8003990 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.705+0000 7f6db97fa640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6da006b2d0 0x7f6da006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.705+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6da8089110 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.705+0000 7f6dc0bd2640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6da006b2d0 0x7f6da006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.706+0000 7f6dc0bd2640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6da006b2d0 0x7f6da006d790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6db0036000 tx=0x7f6db003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.714+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6da8053f40 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.743+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12b0144e60 con 0x7f129c0047c0 2023-12-04T05:46:34.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.743+0000 7f12a2ffd640 1 -- 172.21.15.111:0/2854158647 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12b0144e60 con 0x7f129c0047c0 2023-12-04T05:46:34.745 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:46:34.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f129406b2d0 msgr2=0x7f129406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f129406b2d0 0x7f129406d790 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f12ac033000 tx=0x7f12ac009030 comp rx=0 tx=0).stop 2023-12-04T05:46:34.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 msgr2=0x7f12b013a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f12b013a870 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f12a4001340 tx=0x7f12a40074f0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 shutdown_connections 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f129406b2d0 0x7f129406d790 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b01e4bb0 0x7f12b01e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b0154f10 0x7f12b01e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 --2- 172.21.15.111:0/2854158647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f129c0047c0 0x7f12b013a870 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 >> 172.21.15.111:0/2854158647 conn(0x7f12b00b5500 msgr2=0x7f12b00b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 shutdown_connections 2023-12-04T05:46:34.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.745+0000 7f12b8489640 1 -- 172.21.15.111:0/2854158647 wait complete. 2023-12-04T05:46:34.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:46:34.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.858+0000 7feea7133640 1 Processor -- start 2023-12-04T05:46:34.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.858+0000 7feea7133640 1 -- start start 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea7133640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feea015ed10 0x7feea015f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea7133640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feea00cfaf0 0x7feea00cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea7133640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea013e380 con 0x7feea006e7e0 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea7133640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea015b810 con 0x7feea015ed10 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea7133640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea015baf0 con 0x7feea00cfaf0 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7fee9ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feea015ed10 0x7feea015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea4ecf640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7feea006e7e0 0x7feea006ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38516/0 (socket says 172.21.15.111:38516) 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea4ecf640 1 -- 172.21.15.111:0/2709132765 learned_addr learned my addr 172.21.15.111:0/2709132765 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea56d0640 1 --2- 172.21.15.111:0/2709132765 >> v2:172.21.15.192:3300/0 conn(0x7feea00cfaf0 0x7feea00cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7fee9ffff640 1 -- 172.21.15.111:0/2709132765 >> v2:172.21.15.192:3300/0 conn(0x7feea00cfaf0 msgr2=0x7feea00cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:34.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7fee9ffff640 1 --2- 172.21.15.111:0/2709132765 >> v2:172.21.15.192:3300/0 conn(0x7feea00cfaf0 0x7feea00cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7fee9ffff640 1 -- 172.21.15.111:0/2709132765 >> v1:172.21.15.111:6789/0 conn(0x7feea006e7e0 legacy=0x7feea006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7fee9f7fe640 1 -- 172.21.15.111:0/2709132765 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150536612 0 0) 0x7feea013e380 con 0x7feea006e7e0 2023-12-04T05:46:34.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7fee9ffff640 1 -- 172.21.15.111:0/2709132765 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea015bdd0 con 0x7feea015ed10 2023-12-04T05:46:34.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.859+0000 7feea56d0640 1 --2- 172.21.15.111:0/2709132765 >> v2:172.21.15.192:3300/0 conn(0x7feea00cfaf0 0x7feea00cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:34.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.860+0000 7fee9ffff640 1 --2- 172.21.15.111:0/2709132765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea015ed10 0x7feea015f520 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feea00b0b30 tx=0x7fee8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b00f2e48a91e35f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.860+0000 7fee9f7fe640 1 -- 172.21.15.111:0/2709132765 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c02e070 con 0x7feea015ed10 2023-12-04T05:46:34.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.860+0000 7fee9f7fe640 1 -- 172.21.15.111:0/2709132765 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c0315a0 con 0x7feea015ed10 2023-12-04T05:46:34.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.860+0000 7fee9f7fe640 1 -- 172.21.15.111:0/2709132765 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c031930 con 0x7feea015ed10 2023-12-04T05:46:34.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 -- 172.21.15.111:0/2709132765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea015ed10 msgr2=0x7feea015f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 --2- 172.21.15.111:0/2709132765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea015ed10 0x7feea015f520 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feea00b0b30 tx=0x7fee8c02b840 comp rx=0 tx=0).stop 2023-12-04T05:46:34.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 -- 172.21.15.111:0/2709132765 shutdown_connections 2023-12-04T05:46:34.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 --2- 172.21.15.111:0/2709132765 >> v2:172.21.15.192:3300/0 conn(0x7feea00cfaf0 0x7feea00cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 --2- 172.21.15.111:0/2709132765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea015ed10 0x7feea015f520 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 -- 172.21.15.111:0/2709132765 >> 172.21.15.111:0/2709132765 conn(0x7feea006a8b0 msgr2=0x7feea006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 -- 172.21.15.111:0/2709132765 shutdown_connections 2023-12-04T05:46:34.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.861+0000 7feea7133640 1 -- 172.21.15.111:0/2709132765 wait complete. 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.862+0000 7feea7133640 1 Processor -- start 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.862+0000 7feea7133640 1 -- start start 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea7133640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 0x7feea0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea7133640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feea00cfaf0 0x7feea015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea7133640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea015ed10 0x7feea0153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea7133640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea01535d0 con 0x7feea006e7e0 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea7133640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea01538b0 con 0x7feea015ed10 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea7133640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea0157150 con 0x7feea00cfaf0 2023-12-04T05:46:34.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 0x7feea0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 0x7feea0154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45942/0 (socket says 172.21.15.111:45942) 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 -- 172.21.15.111:0/549188006 learned_addr learned my addr 172.21.15.111:0/549188006 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea56d0640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea015ed10 0x7feea0153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7fee9ffff640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feea00cfaf0 0x7feea015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 -- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feea00cfaf0 msgr2=0x7feea015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feea00cfaf0 0x7feea015e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 -- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea015ed10 msgr2=0x7feea0153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.863+0000 7feea4ecf640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea015ed10 0x7feea0153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7feea4ecf640 1 -- 172.21.15.111:0/549188006 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c009400 con 0x7feea006e7e0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7fee9ffff640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feea00cfaf0 0x7feea015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7feea4ecf640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 0x7feea0154d40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fee9400c4d0 tx=0x7fee9400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee940074e0 con 0x7feea006e7e0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94007660 con 0x7feea006e7e0 2023-12-04T05:46:34.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea0157430 con 0x7feea006e7e0 2023-12-04T05:46:34.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94004a20 con 0x7feea006e7e0 2023-12-04T05:46:34.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.864+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea0155280 con 0x7feea006e7e0 2023-12-04T05:46:34.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.865+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea00693d0 con 0x7feea006e7e0 2023-12-04T05:46:34.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.865+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fee94004b80 con 0x7feea006e7e0 2023-12-04T05:46:34.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.866+0000 7fee9d7fa640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee8406b2a0 0x7fee8406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:34.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.866+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fee94088dd0 con 0x7feea006e7e0 2023-12-04T05:46:34.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.866+0000 7fee9ffff640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee8406b2a0 0x7fee8406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:34.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.866+0000 7fee9ffff640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee8406b2a0 0x7fee8406d760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee8c036000 tx=0x7fee8c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:34.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.870+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee94053be0 con 0x7feea006e7e0 2023-12-04T05:46:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.903+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6dbc0693d0 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.904+0000 7f6db97fa640 1 -- 172.21.15.111:0/643027789 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v49) v1 ==== 80+0+244 (secure 0 0 0) 0x7f6da8057560 con 0x7f6dbc0cfaf0 2023-12-04T05:46:34.906 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6835/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:34.906 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6833/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:34.906 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6836/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:34.906 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6834/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:listed 4 entries 2023-12-04T05:46:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6da006b2d0 msgr2=0x7f6da006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6da006b2d0 0x7f6da006d790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6db0036000 tx=0x7f6db003d040 comp rx=0 tx=0).stop 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 msgr2=0x7f6dbc0b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0b1650 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c400 tx=0x7f6da800c8c0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 shutdown_connections 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f6da006b2d0 0x7f6da006d790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dbc15ed10 0x7f6dbc1396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6dbc0cfaf0 0x7f6dbc0b1650 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 --2- 172.21.15.111:0/643027789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dbc06e7e0 0x7f6dbc158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 >> 172.21.15.111:0/643027789 conn(0x7f6dbc06a8b0 msgr2=0x7f6dbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 shutdown_connections 2023-12-04T05:46:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:34.906+0000 7f6dc2e36640 1 -- 172.21.15.111:0/643027789 wait complete. 2023-12-04T05:46:34.918 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/fuse_mount.py", line 399, in umount_wait self.umount(cleanup=False) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/fuse_mount.py", line 320, in umount if self.is_blocked(): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 197, in is_blocked return self.addr in output TypeError: 'in ' requires string as left operand, not NoneType 2023-12-04T05:46:34.919 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:46:34.919 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:34.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:46:34.935 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi111.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-12-04T05:46:34.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:46:34.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:34.935 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-04T05:46:34.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:46:34.990 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-04T05:46:34.991 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:46:34.991 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:35.010 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:35.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.057+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feea00b0d00 con 0x7feea006e7e0 2023-12-04T05:46:35.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.058+0000 7fee9d7fa640 1 -- 172.21.15.111:0/549188006 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v49) v1 ==== 80+0+244 (secure 0 0 0) 0x7fee94018020 con 0x7feea006e7e0 2023-12-04T05:46:35.059 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6835/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:35.060 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6833/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:35.060 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6836/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:35.060 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6834/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:35.060 INFO:teuthology.orchestra.run.smithi111.stderr:listed 4 entries 2023-12-04T05:46:35.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee8406b2a0 msgr2=0x7fee8406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee8406b2a0 0x7fee8406d760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee8c036000 tx=0x7fee8c03d040 comp rx=0 tx=0).stop 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 msgr2=0x7feea0154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 0x7feea0154d40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fee9400c4d0 tx=0x7fee9400c990 comp rx=0 tx=0).stop 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 shutdown_connections 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fee8406b2a0 0x7fee8406d760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feea015ed10 0x7feea0153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feea00cfaf0 0x7feea015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 --2- 172.21.15.111:0/549188006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea006e7e0 0x7feea0154d40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 >> 172.21.15.111:0/549188006 conn(0x7feea006a8b0 msgr2=0x7feea00d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 shutdown_connections 2023-12-04T05:46:35.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.060+0000 7feea7133640 1 -- 172.21.15.111:0/549188006 wait complete. 2023-12-04T05:46:35.072 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-04T05:46:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:35.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-04T05:46:35.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 Processor -- start 2023-12-04T05:46:35.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 -- start start 2023-12-04T05:46:35.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:46:35.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:35.133 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-04T05:46:35.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc85006c960 0x7fc85015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8500aea20 con 0x7fc85013cee0 2023-12-04T05:46:35.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85013d2c0 con 0x7fc85006c960 2023-12-04T05:46:35.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.130+0000 7fc8559a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85015b3a0 con 0x7fc850140200 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84effd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc85013cee0 0x7fc85006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38524/0 (socket says 172.21.15.111:38524) 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84effd640 1 -- 172.21.15.111:0/3428082658 learned_addr learned my addr 172.21.15.111:0/3428082658 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84f7fe640 1 --2- 172.21.15.111:0/3428082658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc8467fc640 1 --2- 172.21.15.111:0/3428082658 >> v2:172.21.15.192:3300/0 conn(0x7fc85006c960 0x7fc85015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84e7fc640 1 -- 172.21.15.111:0/3428082658 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108257630 0 0) 0x7fc8500aea20 con 0x7fc85013cee0 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84e7fc640 1 -- 172.21.15.111:0/3428082658 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc83c003520 con 0x7fc85013cee0 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84f7fe640 1 -- 172.21.15.111:0/3428082658 >> v2:172.21.15.192:3300/0 conn(0x7fc85006c960 msgr2=0x7fc85015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84f7fe640 1 --2- 172.21.15.111:0/3428082658 >> v2:172.21.15.192:3300/0 conn(0x7fc85006c960 0x7fc85015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84f7fe640 1 -- 172.21.15.111:0/3428082658 >> v1:172.21.15.111:6789/0 conn(0x7fc85013cee0 legacy=0x7fc85006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc84f7fe640 1 -- 172.21.15.111:0/3428082658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85015b0c0 con 0x7fc850140200 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.131+0000 7fc8467fc640 1 --2- 172.21.15.111:0/3428082658 >> v2:172.21.15.192:3300/0 conn(0x7fc85006c960 0x7fc85015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.132+0000 7fc84f7fe640 1 --2- 172.21.15.111:0/3428082658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85015a9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc848009d90 tx=0x7fc84802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9563eeb4d20c041a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.132+0000 7fc84e7fc640 1 -- 172.21.15.111:0/3428082658 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84802cbc0 con 0x7fc850140200 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.132+0000 7fc84e7fc640 1 -- 172.21.15.111:0/3428082658 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84802cd20 con 0x7fc850140200 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.132+0000 7fc84e7fc640 1 -- 172.21.15.111:0/3428082658 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848034640 con 0x7fc850140200 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 -- 172.21.15.111:0/3428082658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 msgr2=0x7fc85015a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 --2- 172.21.15.111:0/3428082658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85015a9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc848009d90 tx=0x7fc84802be10 comp rx=0 tx=0).stop 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 -- 172.21.15.111:0/3428082658 shutdown_connections 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 --2- 172.21.15.111:0/3428082658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85015a9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 --2- 172.21.15.111:0/3428082658 >> v2:172.21.15.192:3300/0 conn(0x7fc85006c960 0x7fc85015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 -- 172.21.15.111:0/3428082658 >> 172.21.15.111:0/3428082658 conn(0x7fc85006a4c0 msgr2=0x7fc85006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 -- 172.21.15.111:0/3428082658 shutdown_connections 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.133+0000 7fc8559a9640 1 -- 172.21.15.111:0/3428082658 wait complete. 2023-12-04T05:46:35.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.134+0000 7fc8559a9640 1 Processor -- start 2023-12-04T05:46:35.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.134+0000 7fc8559a9640 1 -- start start 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 0x7fc8500b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85013cee0 0x7fc85013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc850138a40 con 0x7fc85006c960 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500b3840 con 0x7fc85013cee0 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500b2c00 con 0x7fc850140200 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 0x7fc8500b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 0x7fc8500b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45956/0 (socket says 172.21.15.111:45956) 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 -- 172.21.15.111:0/2094260562 learned_addr learned my addr 172.21.15.111:0/2094260562 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84f7fe640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8467fc640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85013cee0 0x7fc85013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 -- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 msgr2=0x7fc85014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 -- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85013cee0 msgr2=0x7fc85013e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85013cee0 0x7fc85013e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 -- 172.21.15.111:0/2094260562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc838009400 con 0x7fc85006c960 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84effd640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 0x7fc8500b3c60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c430 tx=0x7fc84000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840005ce0 con 0x7fc85006c960 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840005e40 con 0x7fc85006c960 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc848009a70 con 0x7fc85006c960 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.135+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8400072f0 con 0x7fc85006c960 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.136+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8500b3180 con 0x7fc85006c960 2023-12-04T05:46:35.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.136+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8500703d0 con 0x7fc85006c960 2023-12-04T05:46:35.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.137+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fc840007450 con 0x7fc85006c960 2023-12-04T05:46:35.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.137+0000 7fc84cff9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83006b2a0 0x7fc83006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.137+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fc840089290 con 0x7fc85006c960 2023-12-04T05:46:35.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.137+0000 7fc8467fc640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83006b2a0 0x7fc83006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.138+0000 7fc8467fc640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83006b2a0 0x7fc83006d760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc85006c470 tx=0x7fc838009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.141+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8500703d0 con 0x7fc85006c960 2023-12-04T05:46:35.189 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:46:35.189 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:35.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:46:35.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:46:35.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:35.205 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-04T05:46:35.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:46:35.259 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-04T05:46:35.260 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:46:35.260 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:35.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:46:35.275 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:46:35.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:35.276 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-04T05:46:35.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.315+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8500703d0 con 0x7fc85006c960 2023-12-04T05:46:35.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.316+0000 7fc84cff9640 1 -- 172.21.15.111:0/2094260562 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7fc8500703d0 con 0x7fc85006c960 2023-12-04T05:46:35.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:35.318 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:35.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83006b2a0 msgr2=0x7fc83006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83006b2a0 0x7fc83006d760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc85006c470 tx=0x7fc838009190 comp rx=0 tx=0).stop 2023-12-04T05:46:35.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 msgr2=0x7fc8500b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 0x7fc8500b3c60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c430 tx=0x7fc84000c8f0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 shutdown_connections 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fc83006b2a0 0x7fc83006d760 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc850140200 0x7fc85014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85013cee0 0x7fc85013e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 --2- 172.21.15.111:0/2094260562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85006c960 0x7fc8500b3c60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 >> 172.21.15.111:0/2094260562 conn(0x7fc85006a4c0 msgr2=0x7fc850070980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 shutdown_connections 2023-12-04T05:46:35.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.318+0000 7fc8559a9640 1 -- 172.21.15.111:0/2094260562 wait complete. 2023-12-04T05:46:35.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:46:35.330 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-04T05:46:35.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:46:35.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:41:00.843728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:41:01.612301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:46:35.333 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:46:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.472+0000 7fcab48e0640 1 Processor -- start 2023-12-04T05:46:35.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.472+0000 7fcab48e0640 1 -- start start 2023-12-04T05:46:35.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.473+0000 7fcab48e0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab006c960 0x7fcab015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.473+0000 7fcab48e0640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab00aea20 con 0x7fcab013cee0 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.473+0000 7fcab48e0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab013d2c0 con 0x7fcab0140200 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.473+0000 7fcab48e0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab015b3a0 con 0x7fcab006c960 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaae59c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcab013cee0 0x7fcab006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38530/0 (socket says 172.21.15.111:38530) 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaae59c640 1 -- 172.21.15.111:0/514258096 learned_addr learned my addr 172.21.15.111:0/514258096 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaadd9b640 1 --2- 172.21.15.111:0/514258096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab006c960 0x7fcab015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaad59a640 1 -- 172.21.15.111:0/514258096 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766668935 0 0) 0x7fcab00aea20 con 0x7fcab013cee0 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaad59a640 1 -- 172.21.15.111:0/514258096 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca98003520 con 0x7fcab013cee0 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaad59a640 1 -- 172.21.15.111:0/514258096 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602583113 0 0) 0x7fcaa8001490 con 0x7fcab0140200 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaadd9b640 1 -- 172.21.15.111:0/514258096 >> v1:172.21.15.192:6789/0 conn(0x7fcab0140200 legacy=0x7fcab00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaadd9b640 1 -- 172.21.15.111:0/514258096 >> v1:172.21.15.111:6789/0 conn(0x7fcab013cee0 legacy=0x7fcab006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.474+0000 7fcaadd9b640 1 -- 172.21.15.111:0/514258096 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab015b0c0 con 0x7fcab006c960 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.475+0000 7fcaadd9b640 1 --2- 172.21.15.111:0/514258096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab006c960 0x7fcab015a490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40097d0 tx=0x7fcaa402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9b038f21a445a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.475+0000 7fcaad59a640 1 -- 172.21.15.111:0/514258096 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4031800 con 0x7fcab006c960 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.475+0000 7fcaad59a640 1 -- 172.21.15.111:0/514258096 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4031960 con 0x7fcab006c960 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.475+0000 7fcaad59a640 1 -- 172.21.15.111:0/514258096 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4031cf0 con 0x7fcab006c960 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 -- 172.21.15.111:0/514258096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab006c960 msgr2=0x7fcab015a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 --2- 172.21.15.111:0/514258096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab006c960 0x7fcab015a490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40097d0 tx=0x7fcaa402b8a0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 -- 172.21.15.111:0/514258096 shutdown_connections 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 --2- 172.21.15.111:0/514258096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab006c960 0x7fcab015a490 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 -- 172.21.15.111:0/514258096 >> 172.21.15.111:0/514258096 conn(0x7fcab006a4c0 msgr2=0x7fcab006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 -- 172.21.15.111:0/514258096 shutdown_connections 2023-12-04T05:46:35.478 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7fcab48e0640 1 -- 172.21.15.111:0/514258096 wait complete. 2023-12-04T05:46:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7f330a0c9640 1 Processor -- start 2023-12-04T05:46:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.476+0000 7f330a0c9640 1 -- start start 2023-12-04T05:46:35.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7fcab48e0640 1 Processor -- start 2023-12-04T05:46:35.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7fcab48e0640 1 -- start start 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7f330a0c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f3304160f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7f330a0c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33040d01d0 0x7f330415e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7f330a0c9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330415dcb0 con 0x7f330415e680 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7f330a0c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3304161750 con 0x7f33040d01d0 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.477+0000 7f330a0c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3304161a30 con 0x7f3304146be0 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f3302ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33040d01d0 0x7f330415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f3304160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f3304160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.111:43360/0 (socket says 172.21.15.111:43360) 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 -- 172.21.15.111:0/1618528608 learned_addr learned my addr 172.21.15.111:0/1618528608 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 -- 172.21.15.111:0/1618528608 >> v2:172.21.15.192:3300/0 conn(0x7f33040d01d0 msgr2=0x7f330415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 --2- 172.21.15.111:0/1618528608 >> v2:172.21.15.192:3300/0 conn(0x7f33040d01d0 0x7f330415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 -- 172.21.15.111:0/1618528608 >> v1:172.21.15.192:6789/0 conn(0x7f330415e680 legacy=0x7f33040b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f33037fe640 1 -- 172.21.15.111:0/1618528608 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304161d10 con 0x7f3304146be0 2023-12-04T05:46:35.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7f3302ffd640 1 --2- 172.21.15.111:0/1618528608 >> v2:172.21.15.192:3300/0 conn(0x7f33040d01d0 0x7f330415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcab48e0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab006c960 0x7fcab01f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcab48e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 0x7fcab013d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcab48e0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab0140200 0x7fcab00703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcab48e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0138e20 con 0x7fcab013cee0 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcab48e0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab01f4750 con 0x7fcab006c960 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcab48e0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00b24e0 con 0x7fcab0140200 2023-12-04T05:46:35.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 0x7fcab013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 0x7fcab013d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45966/0 (socket says 172.21.15.111:45966) 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 -- 172.21.15.111:0/3883238176 learned_addr learned my addr 172.21.15.111:0/3883238176 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaae59c640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab006c960 0x7fcab01f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaaed9d640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab0140200 0x7fcab00703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 -- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab0140200 msgr2=0x7fcab00703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab0140200 0x7fcab00703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 -- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab006c960 msgr2=0x7fcab01f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab006c960 0x7fcab01f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaadd9b640 1 -- 172.21.15.111:0/3883238176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4009400 con 0x7fcab013cee0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.478+0000 7fcaae59c640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab006c960 0x7fcab01f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7fcaadd9b640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 0x7fcab013d790 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40097d0 tx=0x7fcaa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4004230 con 0x7fcab013cee0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4004390 con 0x7fcab013cee0 2023-12-04T05:46:35.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab01f4980 con 0x7fcab013cee0 2023-12-04T05:46:35.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7f33037fe640 1 --2- 172.21.15.111:0/1618528608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f3304160f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009870 tx=0x7f32f402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3dfb077564651 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa403baa0 con 0x7fcab013cee0 2023-12-04T05:46:35.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab01f4f40 con 0x7fcab013cee0 2023-12-04T05:46:35.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7f33027fc640 1 -- 172.21.15.111:0/1618528608 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4031640 con 0x7f3304146be0 2023-12-04T05:46:35.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7f33027fc640 1 -- 172.21.15.111:0/1618528608 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f40317a0 con 0x7f3304146be0 2023-12-04T05:46:35.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.479+0000 7f33027fc640 1 -- 172.21.15.111:0/1618528608 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4031b30 con 0x7f3304146be0 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 -- 172.21.15.111:0/1618528608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 msgr2=0x7f3304160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1618528608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f3304160f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009870 tx=0x7f32f402b8f0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 -- 172.21.15.111:0/1618528608 shutdown_connections 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1618528608 >> v2:172.21.15.192:3300/0 conn(0x7f33040d01d0 0x7f330415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1618528608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f3304160f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 -- 172.21.15.111:0/1618528608 >> 172.21.15.111:0/1618528608 conn(0x7f33040b5500 msgr2=0x7f33040b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 -- 172.21.15.111:0/1618528608 shutdown_connections 2023-12-04T05:46:35.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab01f5130 con 0x7fcab013cee0 2023-12-04T05:46:35.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7f330a0c9640 1 -- 172.21.15.111:0/1618528608 wait complete. 2023-12-04T05:46:35.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.480+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fcaa4043070 con 0x7fcab013cee0 2023-12-04T05:46:35.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.481+0000 7fca9effd640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fca9006b2a0 0x7fca9006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.481+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcaa402e070 con 0x7fcab013cee0 2023-12-04T05:46:35.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.481+0000 7fcaae59c640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fca9006b2a0 0x7fca9006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.481+0000 7fcaae59c640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fca9006b2a0 0x7fca9006d760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005880 tx=0x7fcaa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.481+0000 7f330a0c9640 1 Processor -- start 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.482+0000 7f330a0c9640 1 -- start start 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f330a0c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33040d01d0 0x7f33040c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f330a0c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f33040c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f330a0c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 0x7f33040c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f330a0c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041f3c40 con 0x7f330415e680 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f330a0c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041f44e0 con 0x7f33040d01d0 2023-12-04T05:46:35.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f330a0c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33040c5d60 con 0x7f3304146be0 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 0x7f33040c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 0x7f33040c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45970/0 (socket says 172.21.15.111:45970) 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 -- 172.21.15.111:0/1136600247 learned_addr learned my addr 172.21.15.111:0/1136600247 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f33037fe640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33040d01d0 0x7f33040c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3302ffd640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f33040c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 -- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 msgr2=0x7f33040c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f33040c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 -- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33040d01d0 msgr2=0x7f33040c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33040d01d0 0x7f33040c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f3303fff640 1 -- 172.21.15.111:0/1136600247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8009400 con 0x7f330415e680 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.483+0000 7f33037fe640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33040d01d0 0x7f33040c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.484+0000 7f3303fff640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 0x7f33040c4fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c730 tx=0x7f32fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.486+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc003b70 con 0x7f330415e680 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.486+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc003cd0 con 0x7f330415e680 2023-12-04T05:46:35.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.486+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00aa90 con 0x7f330415e680 2023-12-04T05:46:35.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.486+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f40094d0 con 0x7f330415e680 2023-12-04T05:46:35.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.486+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaa4076800 con 0x7fcab013cee0 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.486+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33041f4170 con 0x7f330415e680 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.487+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3304067460 con 0x7f330415e680 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.488+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f32fc00abf0 con 0x7f330415e680 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.488+0000 7f32ebfff640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f32f006b2d0 0x7f32f006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.489+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f32fc0899c0 con 0x7f330415e680 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.489+0000 7f33037fe640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f32f006b2d0 0x7f32f006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.489+0000 7f33037fe640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f32f006b2d0 0x7f32f006d790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32f4036000 tx=0x7f32f403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.494+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32fc0547d0 con 0x7f330415e680 2023-12-04T05:46:35.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.667+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33041f4650 con 0x7f330415e680 2023-12-04T05:46:35.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.667+0000 7f32ebfff640 1 -- 172.21.15.111:0/1136600247 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32fc057df0 con 0x7f330415e680 2023-12-04T05:46:35.669 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:46:35.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f32f006b2d0 msgr2=0x7f32f006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f32f006b2d0 0x7f32f006d790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32f4036000 tx=0x7f32f403c040 comp rx=0 tx=0).stop 2023-12-04T05:46:35.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 msgr2=0x7f33040c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 0x7f33040c4fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c730 tx=0x7f32fc00cbf0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 shutdown_connections 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f32f006b2d0 0x7f32f006d790 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330415e680 0x7f33040c4fe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3304146be0 0x7f33040c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 --2- 172.21.15.111:0/1136600247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33040d01d0 0x7f33040c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 >> 172.21.15.111:0/1136600247 conn(0x7f33040b5500 msgr2=0x7f3304071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 shutdown_connections 2023-12-04T05:46:35.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.669+0000 7f330a0c9640 1 -- 172.21.15.111:0/1136600247 wait complete. 2023-12-04T05:46:35.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:46:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.697+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab00693d0 con 0x7fcab013cee0 2023-12-04T05:46:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.698+0000 7fca9effd640 1 -- 172.21.15.111:0/3883238176 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3400 (secure 0 0 0) 0x7fcaa4035020 con 0x7fcab013cee0 2023-12-04T05:46:35.699 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:35.699 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:45:11.358964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5868,"mds_1":5042},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5042":{"gid":5042,"name":"b","rank":1,"incarnation":30,"state":"up:active","state_seq":20,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"laggy_since":"2023-12-04T05:42:22.222606+0000","export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":65,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:46:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 37 2023-12-04T05:46:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fca9006b2a0 msgr2=0x7fca9006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fca9006b2a0 0x7fca9006d760 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005880 tx=0x7fcaa0009040 comp rx=0 tx=0).stop 2023-12-04T05:46:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 msgr2=0x7fcab013d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 0x7fcab013d790 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40097d0 tx=0x7fcaa40046c0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 shutdown_connections 2023-12-04T05:46:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fca9006b2a0 0x7fca9006d760 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.699+0000 7fcab48e0640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab0140200 0x7fcab00703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.700+0000 7fcab48e0640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab013cee0 0x7fcab013d790 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.700+0000 7fcab48e0640 1 --2- 172.21.15.111:0/3883238176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab006c960 0x7fcab01f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.700+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 >> 172.21.15.111:0/3883238176 conn(0x7fcab006a4c0 msgr2=0x7fcab015cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.700+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 shutdown_connections 2023-12-04T05:46:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.700+0000 7fcab48e0640 1 -- 172.21.15.111:0/3883238176 wait complete. 2023-12-04T05:46:35.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:35.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.786+0000 7f74b6743640 1 Processor -- start 2023-12-04T05:46:35.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.786+0000 7f74b6743640 1 -- start start 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b6743640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f74b0144960 0x7f74b006a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b6743640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b6743640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f74b0072440 0x7f74b015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b6743640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b00cc9e0 con 0x7f74b0072440 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b6743640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b00cd280 con 0x7f74b0144960 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b6743640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b00729b0 con 0x7f74b015b770 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f74b0072440 0x7f74b015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f74b0072440 0x7f74b015ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45976/0 (socket says 172.21.15.111:45976) 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 -- 172.21.15.111:0/4175129414 learned_addr learned my addr 172.21.15.111:0/4175129414 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74af7fe640 1 --2- 172.21.15.111:0/4175129414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74affff640 1 --2- 172.21.15.111:0/4175129414 >> v2:172.21.15.192:3300/0 conn(0x7f74b0144960 0x7f74b006a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 -- 172.21.15.111:0/4175129414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 msgr2=0x7f74b006aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:35.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 --2- 172.21.15.111:0/4175129414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b006aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 -- 172.21.15.111:0/4175129414 >> v2:172.21.15.192:3300/0 conn(0x7f74b0144960 msgr2=0x7f74b006a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:35.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 --2- 172.21.15.111:0/4175129414 >> v2:172.21.15.192:3300/0 conn(0x7f74b0144960 0x7f74b006a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.787+0000 7f74b4ce0640 1 -- 172.21.15.111:0/4175129414 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b00ccc30 con 0x7f74b0072440 2023-12-04T05:46:35.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.788+0000 7f74b4ce0640 1 --2- 172.21.15.111:0/4175129414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0072440 0x7f74b015ece0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f74b0074950 tx=0x7f74a802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245cd96e861b0e0a server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.788+0000 7f74aeffd640 1 -- 172.21.15.111:0/4175129414 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a802c9e0 con 0x7f74b0072440 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.788+0000 7f74aeffd640 1 -- 172.21.15.111:0/4175129414 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a802cb40 con 0x7f74b0072440 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.788+0000 7f74aeffd640 1 -- 172.21.15.111:0/4175129414 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80343f0 con 0x7f74b0072440 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 -- 172.21.15.111:0/4175129414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0072440 msgr2=0x7f74b015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 --2- 172.21.15.111:0/4175129414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0072440 0x7f74b015ece0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f74b0074950 tx=0x7f74a802be40 comp rx=0 tx=0).stop 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 -- 172.21.15.111:0/4175129414 shutdown_connections 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 --2- 172.21.15.111:0/4175129414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0072440 0x7f74b015ece0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 --2- 172.21.15.111:0/4175129414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 --2- 172.21.15.111:0/4175129414 >> v2:172.21.15.192:3300/0 conn(0x7f74b0144960 0x7f74b006a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 -- 172.21.15.111:0/4175129414 >> 172.21.15.111:0/4175129414 conn(0x7f74b006d4a0 msgr2=0x7f74b006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 -- 172.21.15.111:0/4175129414 shutdown_connections 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.789+0000 7f74b6743640 1 -- 172.21.15.111:0/4175129414 wait complete. 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.790+0000 7f74b6743640 1 Processor -- start 2023-12-04T05:46:35.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.790+0000 7f74b6743640 1 -- start start 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b6743640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0072440 0x7f74b01f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b6743640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 0x7f74b01f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b6743640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b01f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b6743640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01e2ca0 con 0x7f74b0144960 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b6743640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b013a080 con 0x7f74b0072440 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b6743640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0141d80 con 0x7f74b015b770 2023-12-04T05:46:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 0x7f74b01f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 0x7f74b01f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45990/0 (socket says 172.21.15.111:45990) 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 -- 172.21.15.111:0/1878252224 learned_addr learned my addr 172.21.15.111:0/1878252224 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74affff640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0072440 0x7f74b01f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b4ce0640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b01f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 -- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 msgr2=0x7f74b01f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b01f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 -- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0072440 msgr2=0x7f74b01f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0072440 0x7f74b01f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74af7fe640 1 -- 172.21.15.111:0/1878252224 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8009b30 con 0x7f74b0144960 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.791+0000 7f74b4ce0640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b01f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.792+0000 7f74af7fe640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 0x7f74b01f4ae0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f74a400d800 tx=0x7f74a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.792+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400b660 con 0x7f74b0144960 2023-12-04T05:46:35.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.792+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a400b7c0 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.792+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b014b090 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.792+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400bb50 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.792+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b013d0b0 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.793+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b0067460 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.793+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f74a400bcb0 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.794+0000 7f74acff9640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f749c06b2a0 0x7f749c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.794+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f74a408a200 con 0x7f74b0144960 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.794+0000 7f74affff640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f749c06b2a0 0x7f749c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.794+0000 7f74affff640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f749c06b2a0 0x7f749c06d760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006fd0 tx=0x7f74a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.798+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74a4054f80 con 0x7f74b0144960 2023-12-04T05:46:35.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.854+0000 7f9d1047c640 1 Processor -- start 2023-12-04T05:46:35.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.854+0000 7f9d1047c640 1 -- start start 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.855+0000 7f9d1047c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 0x7f9d080d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.855+0000 7f9d1047c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.855+0000 7f9d1047c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080cdcb0 con 0x7f9d080b20a0 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.855+0000 7f9d1047c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d080d39e0 con 0x7f9d080d1780 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.855+0000 7f9d1047c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080d3cc0 con 0x7f9d080b2ad0 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0e218640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 0x7f9d080d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0ea19640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0da17640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f9d080b20a0 0x7f9d080d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.111:50516/0 (socket says 172.21.15.111:50516) 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0e218640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 0x7f9d080d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:37864/0 (socket says 172.21.15.111:37864) 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0da17640 1 -- 172.21.15.111:0/3003356639 learned_addr learned my addr 172.21.15.111:0/3003356639 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0d216640 1 -- 172.21.15.111:0/3003356639 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376676157 0 0) 0x7f9d080cdcb0 con 0x7f9d080b20a0 2023-12-04T05:46:35.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0ea19640 1 -- 172.21.15.111:0/3003356639 >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 msgr2=0x7f9d080d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:35.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0ea19640 1 --2- 172.21.15.111:0/3003356639 >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 0x7f9d080d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0ea19640 1 -- 172.21.15.111:0/3003356639 >> v1:172.21.15.192:6789/0 conn(0x7f9d080b20a0 legacy=0x7f9d080d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0ea19640 1 -- 172.21.15.111:0/3003356639 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080cdf50 con 0x7f9d080b2ad0 2023-12-04T05:46:35.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.856+0000 7f9d0e218640 1 --2- 172.21.15.111:0/3003356639 >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 0x7f9d080d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.857+0000 7f9d0ea19640 1 --2- 172.21.15.111:0/3003356639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080ce750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009d90 tx=0x7f9d0402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f0d6d04b8d2e17 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.857+0000 7f9d0d216640 1 -- 172.21.15.111:0/3003356639 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0402cbd0 con 0x7f9d080b2ad0 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.857+0000 7f9d0d216640 1 -- 172.21.15.111:0/3003356639 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0402cd30 con 0x7f9d080b2ad0 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.857+0000 7f9d0d216640 1 -- 172.21.15.111:0/3003356639 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0402cea0 con 0x7f9d080b2ad0 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.857+0000 7f9d1047c640 1 -- 172.21.15.111:0/3003356639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 msgr2=0x7f9d080ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.857+0000 7f9d1047c640 1 --2- 172.21.15.111:0/3003356639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080ce750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009d90 tx=0x7f9d0402be20 comp rx=0 tx=0).stop 2023-12-04T05:46:35.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.858+0000 7f9d1047c640 1 -- 172.21.15.111:0/3003356639 shutdown_connections 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.858+0000 7f9d1047c640 1 --2- 172.21.15.111:0/3003356639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080ce750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.858+0000 7f9d1047c640 1 --2- 172.21.15.111:0/3003356639 >> v2:172.21.15.192:3300/0 conn(0x7f9d080d1780 0x7f9d080d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.858+0000 7f9d1047c640 1 -- 172.21.15.111:0/3003356639 >> 172.21.15.111:0/3003356639 conn(0x7f9d0809c4e0 msgr2=0x7f9d080d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.858+0000 7f9d1047c640 1 -- 172.21.15.111:0/3003356639 shutdown_connections 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.858+0000 7f9d1047c640 1 -- 172.21.15.111:0/3003356639 wait complete. 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 Processor -- start 2023-12-04T05:46:35.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 -- start start 2023-12-04T05:46:35.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d080b20a0 0x7f9d0815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 0x7f9d0806c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b8c40 con 0x7f9d080d1780 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080ccf50 con 0x7f9d080b20a0 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.859+0000 7f9d1047c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815a380 con 0x7f9d080b2ad0 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 0x7f9d0806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 0x7f9d0806c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46002/0 (socket says 172.21.15.111:46002) 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 -- 172.21.15.111:0/643627877 learned_addr learned my addr 172.21.15.111:0/643627877 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0da17640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0e218640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d080b20a0 0x7f9d0815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 -- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 msgr2=0x7f9d080b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 -- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d080b20a0 msgr2=0x7f9d0815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d080b20a0 0x7f9d0815a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 -- 172.21.15.111:0/643627877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00009400 con 0x7f9d080d1780 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0e218640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d080b20a0 0x7f9d0815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:35.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d0ea19640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 0x7f9d0806c650 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009eb0 tx=0x7f9d04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0402cea0 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04040650 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04009a70 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04040a00 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.860+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d080cc620 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.861+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d080cd0e0 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.862+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9d04040b60 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.862+0000 7f9cfeffd640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9cec06b2a0 0x7f9cec06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.862+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f9d040ac730 con 0x7f9d080d1780 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.862+0000 7f9d0e218640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9cec06b2a0 0x7f9cec06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:35.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.862+0000 7f9d0e218640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9cec06b2a0 0x7f9cec06d760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009fd0 tx=0x7f9d00009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:35.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.866+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d04077680 con 0x7f9d080d1780 2023-12-04T05:46:35.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.973+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74b015b550 con 0x7f74b0144960 2023-12-04T05:46:35.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.974+0000 7f74acff9640 1 -- 172.21.15.111:0/1878252224 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74a40585a0 con 0x7f74b0144960 2023-12-04T05:46:35.976 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f749c06b2a0 msgr2=0x7f749c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f749c06b2a0 0x7f749c06d760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006fd0 tx=0x7f74a0008040 comp rx=0 tx=0).stop 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 msgr2=0x7f74b01f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 0x7f74b01f4ae0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f74a400d800 tx=0x7f74a400dcc0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 shutdown_connections 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f749c06b2a0 0x7f749c06d760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74b015b770 0x7f74b01f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74b0144960 0x7f74b01f4ae0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 --2- 172.21.15.111:0/1878252224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0072440 0x7f74b01f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:35.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 >> 172.21.15.111:0/1878252224 conn(0x7f74b006d4a0 msgr2=0x7f74b006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:35.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 shutdown_connections 2023-12-04T05:46:35.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:35.976+0000 7f74b6743640 1 -- 172.21.15.111:0/1878252224 wait complete. 2023-12-04T05:46:35.988 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:46:36.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.042+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d080693d0 con 0x7f9d080d1780 2023-12-04T05:46:36.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.044+0000 7f9cfeffd640 1 -- 172.21.15.111:0/643627877 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f9d04009450 con 0x7f9d080d1780 2023-12-04T05:46:36.045 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:36.045 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9cec06b2a0 msgr2=0x7f9cec06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9cec06b2a0 0x7f9cec06d760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009fd0 tx=0x7f9d00009190 comp rx=0 tx=0).stop 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 msgr2=0x7f9d0806c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 0x7f9d0806c650 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009eb0 tx=0x7f9d04004a40 comp rx=0 tx=0).stop 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 shutdown_connections 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9cec06b2a0 0x7f9cec06d760 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d080d1780 0x7f9d0806c650 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d080b2ad0 0x7f9d080b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 --2- 172.21.15.111:0/643627877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d080b20a0 0x7f9d0815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 >> 172.21.15.111:0/643627877 conn(0x7f9d0809c4e0 msgr2=0x7f9d080b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 shutdown_connections 2023-12-04T05:46:36.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.046+0000 7f9d1047c640 1 -- 172.21.15.111:0/643627877 wait complete. 2023-12-04T05:46:36.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-04T05:39:04.516545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-04T05:41:00.843728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-04T05:41:01.612301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-04T05:46:36.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-04T05:46:36.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.132+0000 7f0e4657c640 1 Processor -- start 2023-12-04T05:46:36.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.132+0000 7f0e4657c640 1 -- start start 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e4657c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0e4015dea0 0x7f0e400b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e4657c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e400b2f90 con 0x7f0e4015da60 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e4657c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e400cf280 con 0x7f0e4013d000 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e4657c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e400cf560 con 0x7f0e4015dea0 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e44b19640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0e4015dea0 0x7f0e400b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e44b19640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0e4015dea0 0x7f0e400b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46012/0 (socket says 172.21.15.111:46012) 2023-12-04T05:46:36.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e3f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0e4015da60 0x7f0e400b3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38544/0 (socket says 172.21.15.111:38544) 2023-12-04T05:46:36.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e44b19640 1 -- 172.21.15.111:0/774805819 learned_addr learned my addr 172.21.15.111:0/774805819 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e44b19640 1 -- 172.21.15.111:0/774805819 >> v1:172.21.15.192:6789/0 conn(0x7f0e4013d000 legacy=0x7f0e400ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e3effd640 1 -- 172.21.15.111:0/774805819 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700086403 0 0) 0x7f0e400b2f90 con 0x7f0e4015da60 2023-12-04T05:46:36.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e44b19640 1 -- 172.21.15.111:0/774805819 >> v1:172.21.15.111:6789/0 conn(0x7f0e4015da60 legacy=0x7f0e400b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.137 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.134+0000 7f0e44b19640 1 -- 172.21.15.111:0/774805819 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e400cf840 con 0x7f0e4015dea0 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.135+0000 7f0e44b19640 1 --2- 172.21.15.111:0/774805819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4015dea0 0x7f0e400b3b30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009e50 tx=0x7f0e3802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fb7d387fc97974 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.135+0000 7f0e3effd640 1 -- 172.21.15.111:0/774805819 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3802cbb0 con 0x7f0e4015dea0 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.135+0000 7f0e3effd640 1 -- 172.21.15.111:0/774805819 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3802cd10 con 0x7f0e4015dea0 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.135+0000 7f0e3effd640 1 -- 172.21.15.111:0/774805819 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38034610 con 0x7f0e4015dea0 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 -- 172.21.15.111:0/774805819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4015dea0 msgr2=0x7f0e400b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 --2- 172.21.15.111:0/774805819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4015dea0 0x7f0e400b3b30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009e50 tx=0x7f0e3802c010 comp rx=0 tx=0).stop 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 -- 172.21.15.111:0/774805819 shutdown_connections 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 --2- 172.21.15.111:0/774805819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4015dea0 0x7f0e400b3b30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 -- 172.21.15.111:0/774805819 >> 172.21.15.111:0/774805819 conn(0x7f0e400b1c00 msgr2=0x7f0e400b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.140 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 -- 172.21.15.111:0/774805819 shutdown_connections 2023-12-04T05:46:36.140 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.136+0000 7f0e4657c640 1 -- 172.21.15.111:0/774805819 wait complete. 2023-12-04T05:46:36.140 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.137+0000 7f0e4657c640 1 Processor -- start 2023-12-04T05:46:36.140 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 -- start start 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 0x7f0e401f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4015da60 0x7f0e400b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e4015dea0 0x7f0e400b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e401f5fc0 con 0x7f0e4013d000 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4013e260 con 0x7f0e4015da60 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e4657c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e401573b0 con 0x7f0e4015dea0 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e3ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 0x7f0e401f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e3ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 0x7f0e401f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46020/0 (socket says 172.21.15.111:46020) 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.138+0000 7f0e3ffff640 1 -- 172.21.15.111:0/1462892517 learned_addr learned my addr 172.21.15.111:0/1462892517 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3f7fe640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4015da60 0x7f0e400b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e44b19640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e4015dea0 0x7f0e400b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3ffff640 1 -- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e4015dea0 msgr2=0x7f0e400b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3ffff640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e4015dea0 0x7f0e400b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3ffff640 1 -- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4015da60 msgr2=0x7f0e400b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3ffff640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4015da60 0x7f0e400b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3ffff640 1 -- 172.21.15.111:0/1462892517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38009b30 con 0x7f0e4013d000 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3ffff640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 0x7f0e401f5a80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c400 tx=0x7f0e3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300052d0 con 0x7f0e4013d000 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e30005430 con 0x7f0e4013d000 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4013d4a0 con 0x7f0e4013d000 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30003830 con 0x7f0e4013d000 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.139+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4013da60 con 0x7f0e4013d000 2023-12-04T05:46:36.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.140+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e40067460 con 0x7f0e4013d000 2023-12-04T05:46:36.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.141+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f0e30003990 con 0x7f0e4013d000 2023-12-04T05:46:36.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.141+0000 7f0e3cff9640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e2c06b2d0 0x7f0e2c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.141+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0e300890a0 con 0x7f0e4013d000 2023-12-04T05:46:36.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.141+0000 7f0e3f7fe640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e2c06b2d0 0x7f0e2c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.142+0000 7f0e3f7fe640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e2c06b2d0 0x7f0e2c06d790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0e34007f70 tx=0x7f0e34007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.146+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e30053f60 con 0x7f0e4013d000 2023-12-04T05:46:36.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.206+0000 7fbf89516640 1 Processor -- start 2023-12-04T05:46:36.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.206+0000 7fbf89516640 1 -- start start 2023-12-04T05:46:36.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf89516640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbf840cfaf0 0x7fbf840cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf89516640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 0x7fbf8415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf89516640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf84070190 con 0x7fbf8415e770 2023-12-04T05:46:36.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf89516640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf840cff80 con 0x7fbf840cfaf0 2023-12-04T05:46:36.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf89516640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840703e0 con 0x7fbf840cbf50 2023-12-04T05:46:36.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf82ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbf8415e770 0x7fbf8409a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38548/0 (socket says 172.21.15.111:38548) 2023-12-04T05:46:36.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf82ffd640 1 -- 172.21.15.111:0/3251603425 learned_addr learned my addr 172.21.15.111:0/3251603425 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:36.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188106717 0 0) 0x7fbf84070190 con 0x7fbf8415e770 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf827fc640 1 --2- 172.21.15.111:0/3251603425 >> v2:172.21.15.192:3300/0 conn(0x7fbf840cfaf0 0x7fbf840cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf70003520 con 0x7fbf8415e770 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf837fe640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 0x7fbf8415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 500364123 0 0) 0x7fbf70003520 con 0x7fbf8415e770 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 msgr2=0x7fbf8415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 0x7fbf8415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 >> v2:172.21.15.192:3300/0 conn(0x7fbf840cfaf0 msgr2=0x7fbf840cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 --2- 172.21.15.111:0/3251603425 >> v2:172.21.15.192:3300/0 conn(0x7fbf840cfaf0 0x7fbf840cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf84099ba0 con 0x7fbf8415e770 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7fbf6c001670 con 0x7fbf8415e770 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 >> v1:172.21.15.111:6789/0 conn(0x7fbf8415e770 legacy=0x7fbf8409a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70003970 0x7fbf70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 0x7fbf700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf70004d00 0x7fbf70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840703e0 con 0x7fbf70004d00 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7000fd90 con 0x7fbf70003970 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.207+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7000fed0 con 0x7fbf840cbf50 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf837fe640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf70004d00 0x7fbf70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf82ffd640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70003970 0x7fbf70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf827fc640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 0x7fbf700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf837fe640 1 -- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 msgr2=0x7fbf700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf837fe640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf840cbf50 0x7fbf700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf837fe640 1 -- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70003970 msgr2=0x7fbf70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf837fe640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf70003970 0x7fbf70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.208+0000 7fbf837fe640 1 -- 172.21.15.111:0/3251603425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70003520 con 0x7fbf70004d00 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf837fe640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf70004d00 0x7fbf70005180 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009fc0 tx=0x7fbf700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e4b1d8d320ebed server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c02cc20 con 0x7fbf70004d00 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00f040 con 0x7fbf70004d00 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf81ffb640 1 -- 172.21.15.111:0/3251603425 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c034440 con 0x7fbf70004d00 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf89516640 1 -- 172.21.15.111:0/3251603425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf70004d00 msgr2=0x7fbf70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf89516640 1 --2- 172.21.15.111:0/3251603425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf70004d00 0x7fbf70005180 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009fc0 tx=0x7fbf700057f0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf837fe640 1 -- 172.21.15.111:0/3251603425 reap_dead start 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf89516640 1 -- 172.21.15.111:0/3251603425 shutdown_connections 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf89516640 1 -- 172.21.15.111:0/3251603425 >> 172.21.15.111:0/3251603425 conn(0x7fbf840b2ad0 msgr2=0x7fbf840b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf89516640 1 -- 172.21.15.111:0/3251603425 shutdown_connections 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.209+0000 7fbf89516640 1 -- 172.21.15.111:0/3251603425 wait complete. 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 Processor -- start 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 -- start start 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf84158680 0x7fbf8415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf840b3db0 0x7fbf8406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 0x7fbf840c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8406bf90 con 0x7fbf8406bb10 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8415b190 con 0x7fbf840b3db0 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf89516640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8415a7c0 con 0x7fbf84158680 2023-12-04T05:46:36.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.211+0000 7fbf837fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 0x7fbf840c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 0x7fbf840c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46038/0 (socket says 172.21.15.111:46038) 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 -- 172.21.15.111:0/1084832256 learned_addr learned my addr 172.21.15.111:0/1084832256 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf82ffd640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf84158680 0x7fbf8415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf827fc640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf840b3db0 0x7fbf8406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 -- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf84158680 msgr2=0x7fbf8415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf84158680 0x7fbf8415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 -- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf840b3db0 msgr2=0x7fbf8406b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf840b3db0 0x7fbf8406b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 -- 172.21.15.111:0/1084832256 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf841598d0 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf837fe640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 0x7fbf840c80b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c02c690 tx=0x7fbf7c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c043070 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c003990 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf84159bb0 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c003d40 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.212+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8415a170 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.213+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf840671a0 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.213+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fbf7c034900 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.214+0000 7fbf7b7fe640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf6406b2d0 0x7fbf6406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.214+0000 7fbf82ffd640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf6406b2d0 0x7fbf6406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.214+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fbf7c0ab210 con 0x7fbf8406bb10 2023-12-04T05:46:36.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.214+0000 7fbf82ffd640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf6406b2d0 0x7fbf6406d790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c004840 tx=0x7fbf6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.218+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf7c076020 con 0x7fbf8406bb10 2023-12-04T05:46:36.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.321+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e4006d650 con 0x7f0e4013d000 2023-12-04T05:46:36.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.321+0000 7f0e3cff9640 1 -- 172.21.15.111:0/1462892517 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e30057580 con 0x7f0e4013d000 2023-12-04T05:46:36.323 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:46:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.323+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e2c06b2d0 msgr2=0x7f0e2c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.323+0000 7f0e4657c640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e2c06b2d0 0x7f0e2c06d790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0e34007f70 tx=0x7f0e34007f00 comp rx=0 tx=0).stop 2023-12-04T05:46:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.323+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 msgr2=0x7f0e401f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.323+0000 7f0e4657c640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 0x7f0e401f5a80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c400 tx=0x7f0e3000c8c0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 shutdown_connections 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f0e2c06b2d0 0x7f0e2c06d790 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e4015dea0 0x7f0e400b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4015da60 0x7f0e400b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 --2- 172.21.15.111:0/1462892517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e4013d000 0x7f0e401f5a80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 >> 172.21.15.111:0/1462892517 conn(0x7f0e400b1c00 msgr2=0x7f0e4015a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 shutdown_connections 2023-12-04T05:46:36.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.324+0000 7f0e4657c640 1 -- 172.21.15.111:0/1462892517 wait complete. 2023-12-04T05:46:36.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-04T05:46:36.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.392+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf84067460 con 0x7fbf8406bb10 2023-12-04T05:46:36.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.393+0000 7fbf7b7fe640 1 -- 172.21.15.111:0/1084832256 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf7c079640 con 0x7fbf8406bb10 2023-12-04T05:46:36.395 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-12-04T05:46:36.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf6406b2d0 msgr2=0x7fbf6406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf6406b2d0 0x7fbf6406d790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c004840 tx=0x7fbf6c009040 comp rx=0 tx=0).stop 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 msgr2=0x7fbf840c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 0x7fbf840c80b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c02c690 tx=0x7fbf7c009750 comp rx=0 tx=0).stop 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 shutdown_connections 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fbf6406b2d0 0x7fbf6406d790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf8406bb10 0x7fbf840c80b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf840b3db0 0x7fbf8406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 --2- 172.21.15.111:0/1084832256 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf84158680 0x7fbf8415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 >> 172.21.15.111:0/1084832256 conn(0x7fbf840b2ad0 msgr2=0x7fbf840b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 shutdown_connections 2023-12-04T05:46:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.395+0000 7fbf89516640 1 -- 172.21.15.111:0/1084832256 wait complete. 2023-12-04T05:46:36.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-04T05:46:36.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 Processor -- start 2023-12-04T05:46:36.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 -- start start 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f787413cd00 0x7f787406f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787413d0e0 0x7f78740714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7874140570 con 0x7f7874071a10 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7874071e80 con 0x7f787413cd00 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.478+0000 7f787986b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7874140290 con 0x7f787413d0e0 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f78737fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f7874071a10 0x7f787413f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38556/0 (socket says 172.21.15.111:38556) 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f78737fe640 1 -- 172.21.15.111:0/1124421544 learned_addr learned my addr 172.21.15.111:0/1124421544 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667988158 0 0) 0x7f7874140570 con 0x7f7874071a10 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f78727fc640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787413d0e0 0x7f78740714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c003520 con 0x7f7874071a10 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7872ffd640 1 --2- 172.21.15.111:0/1124421544 >> v2:172.21.15.192:3300/0 conn(0x7f787413cd00 0x7f787406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406368039 0 0) 0x7f785c003520 con 0x7f7874071a10 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787413d0e0 msgr2=0x7f78740714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787413d0e0 0x7f78740714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 >> v2:172.21.15.192:3300/0 conn(0x7f787413cd00 msgr2=0x7f787406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 --2- 172.21.15.111:0/1124421544 >> v2:172.21.15.192:3300/0 conn(0x7f787413cd00 0x7f787406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787406eb30 con 0x7f7874071a10 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f786c0019f0 con 0x7f7874071a10 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 >> v1:172.21.15.111:6789/0 conn(0x7f7874071a10 legacy=0x7f787413f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785c003970 0x7f785c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f785c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c004e10 0x7f785c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7874071e80 con 0x7f785c0042d0 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7874140290 con 0x7f785c004e10 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c00bda0 con 0x7f785c003970 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f78727fc640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f785c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f7872ffd640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785c003970 0x7f785c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.479+0000 7f78737fe640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c004e10 0x7f785c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f78727fc640 1 -- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785c003970 msgr2=0x7f785c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f78727fc640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785c003970 0x7f785c003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f78727fc640 1 -- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c004e10 msgr2=0x7f785c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f78727fc640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c004e10 0x7f785c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f78727fc640 1 -- 172.21.15.111:0/1124421544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c003520 con 0x7f785c0042d0 2023-12-04T05:46:36.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f78727fc640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f785c0047c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7868009ac0 tx=0x7f785c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551d5ba520634a99 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.480+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786802c6b0 con 0x7f785c0042d0 2023-12-04T05:46:36.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786802c810 con 0x7f785c0042d0 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f7871ffb640 1 -- 172.21.15.111:0/1124421544 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786802cba0 con 0x7f785c0042d0 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f787986b640 1 -- 172.21.15.111:0/1124421544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 msgr2=0x7f785c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f787986b640 1 --2- 172.21.15.111:0/1124421544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f785c0047c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7868009ac0 tx=0x7f785c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f78737fe640 1 -- 172.21.15.111:0/1124421544 reap_dead start 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f787986b640 1 -- 172.21.15.111:0/1124421544 shutdown_connections 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f787986b640 1 -- 172.21.15.111:0/1124421544 >> 172.21.15.111:0/1124421544 conn(0x7f78740b1de0 msgr2=0x7f78740b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f787986b640 1 -- 172.21.15.111:0/1124421544 shutdown_connections 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.481+0000 7f787986b640 1 -- 172.21.15.111:0/1124421544 wait complete. 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.482+0000 7f787986b640 1 Processor -- start 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 -- start start 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f7874138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7874141b40 0x7f787415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787415b480 0x7f78741ee730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787413ab70 con 0x7f785c0042d0 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7874138e70 con 0x7f7874141b40 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f787986b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787415b990 con 0x7f787415b480 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f7872ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f7874138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f7872ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f7874138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46058/0 (socket says 172.21.15.111:46058) 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.483+0000 7f7872ffd640 1 -- 172.21.15.111:0/3228794927 learned_addr learned my addr 172.21.15.111:0/3228794927 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f7872ffd640 1 -- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787415b480 msgr2=0x7f78741ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f7872ffd640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787415b480 0x7f78741ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f7872ffd640 1 -- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7874141b40 msgr2=0x7f787415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f78727fc640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7874141b40 0x7f787415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f7872ffd640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7874141b40 0x7f787415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f7872ffd640 1 -- 172.21.15.111:0/3228794927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787413a240 con 0x7f785c0042d0 2023-12-04T05:46:36.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f7872ffd640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f7874138930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7864011a70 tx=0x7f7864011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864009680 con 0x7f785c0042d0 2023-12-04T05:46:36.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400fb50 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787413a520 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400fee0 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.484+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78740b1820 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.485+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7874067460 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.486+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f7864014440 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.486+0000 7f785bfff640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f786006b2c0 0x7f786006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.486+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f786408a5a0 con 0x7f785c0042d0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.486+0000 7f78727fc640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f786006b2c0 0x7f786006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.486+0000 7f78727fc640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f786006b2c0 0x7f786006d780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7868036000 tx=0x7f7868030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.490+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7864055320 con 0x7f785c0042d0 2023-12-04T05:46:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.552+0000 7fd7c3894640 1 Processor -- start 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c3894640 1 -- start start 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c3894640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc099a00 0x7fd7bc06f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c3894640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7bc0cc3a0 0x7fd7bc0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c3894640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0cfd60 con 0x7fd7bc070110 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c3894640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7bc09a5d0 con 0x7fd7bc0cc3a0 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c3894640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc09a8b0 con 0x7fd7bc099a00 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.553+0000 7fd7c1e31640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7bc0cc3a0 0x7fd7bc0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1e31640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7bc0cc3a0 0x7fd7bc0cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46064/0 (socket says 172.21.15.111:46064) 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1e31640 1 -- 172.21.15.111:0/2420249080 learned_addr learned my addr 172.21.15.111:0/2420249080 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1630640 1 --2- 172.21.15.111:0/2420249080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc099a00 0x7fd7bc06f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1e31640 1 -- 172.21.15.111:0/2420249080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc099a00 msgr2=0x7fd7bc06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1e31640 1 --2- 172.21.15.111:0/2420249080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc099a00 0x7fd7bc06f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1e31640 1 -- 172.21.15.111:0/2420249080 >> v1:172.21.15.192:6789/0 conn(0x7fd7bc070110 legacy=0x7fd7bc099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.554+0000 7fd7c1e31640 1 -- 172.21.15.111:0/2420249080 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc0cf560 con 0x7fd7bc0cc3a0 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.555+0000 7fd7c1e31640 1 --2- 172.21.15.111:0/2420249080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0cc830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009e50 tx=0x7fd7b802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89f19ff484e0cec server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.555+0000 7fd7b3fff640 1 -- 172.21.15.111:0/2420249080 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b802cb00 con 0x7fd7bc0cc3a0 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.555+0000 7fd7b3fff640 1 -- 172.21.15.111:0/2420249080 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b802cc60 con 0x7fd7bc0cc3a0 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.555+0000 7fd7b3fff640 1 -- 172.21.15.111:0/2420249080 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80343f0 con 0x7fd7bc0cc3a0 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 -- 172.21.15.111:0/2420249080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc0cc3a0 msgr2=0x7fd7bc0cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 --2- 172.21.15.111:0/2420249080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0cc830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009e50 tx=0x7fd7b802bf60 comp rx=0 tx=0).stop 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 -- 172.21.15.111:0/2420249080 shutdown_connections 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 --2- 172.21.15.111:0/2420249080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0cc830 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 --2- 172.21.15.111:0/2420249080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc099a00 0x7fd7bc06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 -- 172.21.15.111:0/2420249080 >> 172.21.15.111:0/2420249080 conn(0x7fd7bc0b2a70 msgr2=0x7fd7bc0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 -- 172.21.15.111:0/2420249080 shutdown_connections 2023-12-04T05:46:36.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.556+0000 7fd7c3894640 1 -- 172.21.15.111:0/2420249080 wait complete. 2023-12-04T05:46:36.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.557+0000 7fd7c3894640 1 Processor -- start 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.557+0000 7fd7c3894640 1 -- start start 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c3894640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7bc070110 0x7fd7bc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c3894640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 0x7fd7bc0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c3894640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c3894640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c8050 con 0x7fd7bc099a00 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c3894640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c6bf0 con 0x7fd7bc070110 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c3894640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0c6ed0 con 0x7fd7bc0cc3a0 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 0x7fd7bc0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 0x7fd7bc0cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46068/0 (socket says 172.21.15.111:46068) 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 -- 172.21.15.111:0/3331934526 learned_addr learned my addr 172.21.15.111:0/3331934526 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c1630640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7bc070110 0x7fd7bc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c1e31640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 -- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc0cc3a0 msgr2=0x7fd7bc0c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 -- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7bc070110 msgr2=0x7fd7bc153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7bc070110 0x7fd7bc153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 -- 172.21.15.111:0/3331934526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8009b30 con 0x7fd7bc099a00 2023-12-04T05:46:36.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7c0e2f640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 0x7fd7bc0cab80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00c450 tx=0x7fd7ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0052d0 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.558+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac005430 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.559+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc0c71b0 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.559+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac012870 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.559+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc153ca0 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.560+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc0c84e0 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.560+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fd7ac0129d0 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.560+0000 7fd7b1ffb640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd7a006b2d0 0x7fd7a006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.561+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd7ac0896c0 con 0x7fd7bc099a00 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.561+0000 7fd7c1630640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd7a006b2d0 0x7fd7a006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.561+0000 7fd7c1630640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd7a006b2d0 0x7fd7a006d790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009b40 tx=0x7fd7b4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.569+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7bc0c84e0 con 0x7fd7bc099a00 2023-12-04T05:46:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.682+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f787415d820 con 0x7f785c0042d0 2023-12-04T05:46:36.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.683+0000 7f785bfff640 1 -- 172.21.15.111:0/3228794927 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v49) v1 ==== 80+0+244 (secure 0 0 0) 0x7f7864058940 con 0x7f785c0042d0 2023-12-04T05:46:36.686 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6835/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:36.686 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.192:6833/956157809 2023-12-05T05:42:02.218443+0000 2023-12-04T05:46:36.687 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6836/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:36.687 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6834/619465105 2023-12-05T05:41:42.214046+0000 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:listed 4 entries 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f786006b2c0 msgr2=0x7f786006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f786006b2c0 0x7f786006d780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7868036000 tx=0x7f7868030000 comp rx=0 tx=0).stop 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 msgr2=0x7f7874138930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f7874138930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7864011a70 tx=0x7f7864011f30 comp rx=0 tx=0).stop 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 shutdown_connections 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f786006b2c0 0x7f786006d780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f787415b480 0x7f78741ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7874141b40 0x7f787415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 --2- 172.21.15.111:0/3228794927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f785c0042d0 0x7f7874138930 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 >> 172.21.15.111:0/3228794927 conn(0x7f78740b1de0 msgr2=0x7f787406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 shutdown_connections 2023-12-04T05:46:36.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.685+0000 7f787986b640 1 -- 172.21.15.111:0/3228794927 wait complete. 2023-12-04T05:46:36.697 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/fuse_mount.py", line 392, in umount_wait if self.is_blocked(): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 197, in is_blocked return self.addr in output TypeError: 'in ' requires string as left operand, not NoneType 2023-12-04T05:46:36.698 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:46:36.698 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:46:36.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:46:36.716 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-12-04T05:46:36.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:46:36.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:46:36.717 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-04T05:46:36.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.742+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7bc0c84e0 con 0x7fd7bc099a00 2023-12-04T05:46:36.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.742+0000 7fd7b1ffb640 1 -- 172.21.15.111:0/3331934526 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7bc0c84e0 con 0x7fd7bc099a00 2023-12-04T05:46:36.744 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-12-04T05:46:36.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd7a006b2d0 msgr2=0x7fd7a006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd7a006b2d0 0x7fd7a006d790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009b40 tx=0x7fd7b4009190 comp rx=0 tx=0).stop 2023-12-04T05:46:36.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 msgr2=0x7fd7bc0cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 0x7fd7bc0cab80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00c450 tx=0x7fd7ac00c910 comp rx=0 tx=0).stop 2023-12-04T05:46:36.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 shutdown_connections 2023-12-04T05:46:36.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fd7a006b2d0 0x7fd7a006d790 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7bc0cc3a0 0x7fd7bc0c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7bc099a00 0x7fd7bc0cab80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 --2- 172.21.15.111:0/3331934526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7bc070110 0x7fd7bc153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 >> 172.21.15.111:0/3331934526 conn(0x7fd7bc0b2a70 msgr2=0x7fd7bc0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 shutdown_connections 2023-12-04T05:46:36.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.744+0000 7fd7c3894640 1 -- 172.21.15.111:0/3331934526 wait complete. 2023-12-04T05:46:36.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-04T05:46:36.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:46:36.775 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-04T05:46:36.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.858+0000 7fcf35b5a640 1 Processor -- start 2023-12-04T05:46:36.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.858+0000 7fcf35b5a640 1 -- start start 2023-12-04T05:46:36.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.859+0000 7fcf35b5a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcf3015b0c0 0x7fcf3015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.859+0000 7fcf35b5a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcf30072250 0x7fcf300cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.859+0000 7fcf35b5a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf300cd320 con 0x7fcf300ce940 2023-12-04T05:46:36.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.859+0000 7fcf35b5a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf300ced80 con 0x7fcf3015b0c0 2023-12-04T05:46:36.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.859+0000 7fcf35b5a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf300cbee0 con 0x7fcf30072250 2023-12-04T05:46:36.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.859+0000 7fcf2f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcf3015b0c0 0x7fcf3015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2effd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcf300ce940 0x7fcf30071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38564/0 (socket says 172.21.15.111:38564) 2023-12-04T05:46:36.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2effd640 1 -- 172.21.15.111:0/2355565364 learned_addr learned my addr 172.21.15.111:0/2355565364 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:36.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcf3015b0c0 0x7fcf3015ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46080/0 (socket says 172.21.15.111:46080) 2023-12-04T05:46:36.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2ffff640 1 --2- 172.21.15.111:0/2355565364 >> v2:172.21.15.192:3300/0 conn(0x7fcf30072250 0x7fcf300cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/2355565364 >> v2:172.21.15.192:3300/0 conn(0x7fcf30072250 msgr2=0x7fcf300cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2f7fe640 1 --2- 172.21.15.111:0/2355565364 >> v2:172.21.15.192:3300/0 conn(0x7fcf30072250 0x7fcf300cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/2355565364 >> v1:172.21.15.111:6789/0 conn(0x7fcf300ce940 legacy=0x7fcf30071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/2355565364 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300cc710 con 0x7fcf3015b0c0 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2e7fc640 1 -- 172.21.15.111:0/2355565364 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957324511 0 0) 0x7fcf300cd320 con 0x7fcf300ce940 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.860+0000 7fcf2ffff640 1 --2- 172.21.15.111:0/2355565364 >> v2:172.21.15.192:3300/0 conn(0x7fcf30072250 0x7fcf300cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf2f7fe640 1 --2- 172.21.15.111:0/2355565364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf3015b0c0 0x7fcf3015ed10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fcf3006e850 tx=0x7fcf2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a03893d54418735 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf2e7fc640 1 -- 172.21.15.111:0/2355565364 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2002e070 con 0x7fcf3015b0c0 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf2e7fc640 1 -- 172.21.15.111:0/2355565364 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf200315a0 con 0x7fcf3015b0c0 2023-12-04T05:46:36.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf2e7fc640 1 -- 172.21.15.111:0/2355565364 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20031930 con 0x7fcf3015b0c0 2023-12-04T05:46:36.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 -- 172.21.15.111:0/2355565364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf3015b0c0 msgr2=0x7fcf3015ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/2355565364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf3015b0c0 0x7fcf3015ed10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fcf3006e850 tx=0x7fcf2002b840 comp rx=0 tx=0).stop 2023-12-04T05:46:36.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 -- 172.21.15.111:0/2355565364 shutdown_connections 2023-12-04T05:46:36.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/2355565364 >> v2:172.21.15.192:3300/0 conn(0x7fcf30072250 0x7fcf300cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/2355565364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf3015b0c0 0x7fcf3015ed10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 -- 172.21.15.111:0/2355565364 >> 172.21.15.111:0/2355565364 conn(0x7fcf300b0cf0 msgr2=0x7fcf300b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 -- 172.21.15.111:0/2355565364 shutdown_connections 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.861+0000 7fcf35b5a640 1 -- 172.21.15.111:0/2355565364 wait complete. 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.862+0000 7fcf35b5a640 1 Processor -- start 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 -- start start 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 0x7fcf30159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf300ce940 0x7fcf30154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf3015b0c0 0x7fcf300a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf30158830 con 0x7fcf30072250 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf30154ec0 con 0x7fcf3015b0c0 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf35b5a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf30157f20 con 0x7fcf300ce940 2023-12-04T05:46:36.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf2f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 0x7fcf30159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf2f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 0x7fcf30159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46092/0 (socket says 172.21.15.111:46092) 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.863+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/3183861173 learned_addr learned my addr 172.21.15.111:0/3183861173 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2effd640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf300ce940 0x7fcf30154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2ffff640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf3015b0c0 0x7fcf300a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf300ce940 msgr2=0x7fcf30154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2f7fe640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf300ce940 0x7fcf30154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf3015b0c0 msgr2=0x7fcf300a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2f7fe640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf3015b0c0 0x7fcf300a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2f7fe640 1 -- 172.21.15.111:0/3183861173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf20009400 con 0x7fcf30072250 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf2f7fe640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 0x7fcf30159fc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009890 tx=0x7fcf200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20032900 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20032a60 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf300aaf30 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20032e10 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.864+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf30158450 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.865+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf30069360 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.865+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fcf2003b3f0 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.866+0000 7fcf17fff640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcf1c06b2a0 0x7fcf1c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.866+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fcf200abd90 con 0x7fcf30072250 2023-12-04T05:46:36.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.866+0000 7fcf2effd640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcf1c06b2a0 0x7fcf1c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:36.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.866+0000 7fcf2effd640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcf1c06b2a0 0x7fcf1c06d760 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf24007fd0 tx=0x7fcf24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:36.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:36.870+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf20076ba0 con 0x7fcf30072250 2023-12-04T05:46:37.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.039+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf300b1990 con 0x7fcf30072250 2023-12-04T05:46:37.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.039+0000 7fcf17fff640 1 -- 172.21.15.111:0/3183861173 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf2007a1c0 con 0x7fcf30072250 2023-12-04T05:46:37.041 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-12-04T05:46:37.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcf1c06b2a0 msgr2=0x7fcf1c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcf1c06b2a0 0x7fcf1c06d760 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf24007fd0 tx=0x7fcf24009040 comp rx=0 tx=0).stop 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 msgr2=0x7fcf30159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 0x7fcf30159fc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009890 tx=0x7fcf200046c0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 shutdown_connections 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fcf1c06b2a0 0x7fcf1c06d760 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf3015b0c0 0x7fcf300a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf300ce940 0x7fcf30154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 --2- 172.21.15.111:0/3183861173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf30072250 0x7fcf30159fc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 >> 172.21.15.111:0/3183861173 conn(0x7fcf300b0cf0 msgr2=0x7fcf3015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 shutdown_connections 2023-12-04T05:46:37.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.041+0000 7fcf35b5a640 1 -- 172.21.15.111:0/3183861173 wait complete. 2023-12-04T05:46:37.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:46:37.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 Processor -- start 2023-12-04T05:46:37.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 -- start start 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c071500 0x7ffb5c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffb5c15e560 0x7ffb5c06a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c15db90 con 0x7ffb5c146a20 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c15d840 con 0x7ffb5c15e560 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.152+0000 7ffb617e3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0b38f0 con 0x7ffb5c071500 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffb5c15e560 0x7ffb5c06a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5affd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ffb5c146a20 0x7ffb5c0b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38566/0 (socket says 172.21.15.111:38566) 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5affd640 1 -- 172.21.15.111:0/1270197609 learned_addr learned my addr 172.21.15.111:0/1270197609 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5a7fc640 1 --2- 172.21.15.111:0/1270197609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c071500 0x7ffb5c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5b7fe640 1 -- 172.21.15.111:0/1270197609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c071500 msgr2=0x7ffb5c15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5b7fe640 1 --2- 172.21.15.111:0/1270197609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c071500 0x7ffb5c15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5b7fe640 1 -- 172.21.15.111:0/1270197609 >> v1:172.21.15.111:6789/0 conn(0x7ffb5c146a20 legacy=0x7ffb5c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb5b7fe640 1 -- 172.21.15.111:0/1270197609 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c0b3bd0 con 0x7ffb5c15e560 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.153+0000 7ffb59ffb640 1 -- 172.21.15.111:0/1270197609 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666802726 0 0) 0x7ffb5c15db90 con 0x7ffb5c146a20 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb5b7fe640 1 --2- 172.21.15.111:0/1270197609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c15e560 0x7ffb5c06a4c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097a0 tx=0x7ffb4c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356cd00f1fec8b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb59ffb640 1 -- 172.21.15.111:0/1270197609 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c02e070 con 0x7ffb5c15e560 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb59ffb640 1 -- 172.21.15.111:0/1270197609 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c0315a0 con 0x7ffb5c15e560 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb59ffb640 1 -- 172.21.15.111:0/1270197609 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c031930 con 0x7ffb5c15e560 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 -- 172.21.15.111:0/1270197609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c15e560 msgr2=0x7ffb5c06a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 --2- 172.21.15.111:0/1270197609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c15e560 0x7ffb5c06a4c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097a0 tx=0x7ffb4c02b840 comp rx=0 tx=0).stop 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 -- 172.21.15.111:0/1270197609 shutdown_connections 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 --2- 172.21.15.111:0/1270197609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c15e560 0x7ffb5c06a4c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 --2- 172.21.15.111:0/1270197609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c071500 0x7ffb5c15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 -- 172.21.15.111:0/1270197609 >> 172.21.15.111:0/1270197609 conn(0x7ffb5c06b5d0 msgr2=0x7ffb5c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 -- 172.21.15.111:0/1270197609 shutdown_connections 2023-12-04T05:46:37.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.154+0000 7ffb617e3640 1 -- 172.21.15.111:0/1270197609 wait complete. 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 Processor -- start 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 -- start start 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 0x7ffb5c13dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c146a20 0x7ffb5c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb5c15c670 0x7ffb5c072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c140e90 con 0x7ffb5c071500 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c13dff0 con 0x7ffb5c15c670 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.156+0000 7ffb617e3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c140210 con 0x7ffb5c146a20 2023-12-04T05:46:37.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 0x7ffb5c13dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 0x7ffb5c13dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46106/0 (socket says 172.21.15.111:46106) 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 -- 172.21.15.111:0/3353331681 learned_addr learned my addr 172.21.15.111:0/3353331681 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5a7fc640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c146a20 0x7ffb5c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5b7fe640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb5c15c670 0x7ffb5c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 -- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c146a20 msgr2=0x7ffb5c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c146a20 0x7ffb5c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 -- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb5c15c670 msgr2=0x7ffb5c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb5c15c670 0x7ffb5c072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 -- 172.21.15.111:0/3353331681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c009400 con 0x7ffb5c071500 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb5affd640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 0x7ffb5c13dab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c890 tx=0x7ffb5400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400a5e0 con 0x7ffb5c071500 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5400a760 con 0x7ffb5c071500 2023-12-04T05:46:37.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400ab10 con 0x7ffb5c071500 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c1404f0 con 0x7ffb5c071500 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.157+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c140ab0 con 0x7ffb5c071500 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.158+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c067980 con 0x7ffb5c071500 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.159+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7ffb5400ac70 con 0x7ffb5c071500 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.159+0000 7ffb537fe640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb3c06b2a0 0x7ffb3c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.159+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ffb54089070 con 0x7ffb5c071500 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.159+0000 7ffb5a7fc640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb3c06b2a0 0x7ffb3c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.159+0000 7ffb5a7fc640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb3c06b2a0 0x7ffb3c06d760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0cf0f0 tx=0x7ffb44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.163+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb54053f30 con 0x7ffb5c071500 2023-12-04T05:46:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.364+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5c0693d0 con 0x7ffb5c071500 2023-12-04T05:46:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.365+0000 7ffb537fe640 1 -- 172.21.15.111:0/3353331681 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3400 (secure 0 0 0) 0x7ffb54018090 con 0x7ffb5c071500 2023-12-04T05:46:37.367 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:37.367 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:45:11.358964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5868,"mds_1":5042},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5042":{"gid":5042,"name":"b","rank":1,"incarnation":30,"state":"up:active","state_seq":20,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"laggy_since":"2023-12-04T05:42:22.222606+0000","export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":65,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:46:37.367 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 37 2023-12-04T05:46:37.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb3c06b2a0 msgr2=0x7ffb3c06d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb3c06b2a0 0x7ffb3c06d760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0cf0f0 tx=0x7ffb44008040 comp rx=0 tx=0).stop 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 msgr2=0x7ffb5c13dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 0x7ffb5c13dab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c890 tx=0x7ffb5400cd50 comp rx=0 tx=0).stop 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 shutdown_connections 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7ffb3c06b2a0 0x7ffb3c06d760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb5c15c670 0x7ffb5c072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb5c146a20 0x7ffb5c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 --2- 172.21.15.111:0/3353331681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c071500 0x7ffb5c13dab0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 >> 172.21.15.111:0/3353331681 conn(0x7ffb5c06b5d0 msgr2=0x7ffb5c0b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 shutdown_connections 2023-12-04T05:46:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.367+0000 7ffb617e3640 1 -- 172.21.15.111:0/3353331681 wait complete. 2023-12-04T05:46:37.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-04T05:46:37.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.478+0000 7f9f5851d640 1 Processor -- start 2023-12-04T05:46:37.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.478+0000 7f9f5851d640 1 -- start start 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f5851d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 0x7f9f5006c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f5851d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f5015aba0 0x7f9f5006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f5851d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f500aea20 con 0x7f9f50140200 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f5851d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5013d2c0 con 0x7f9f5015aba0 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f5851d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015b3a0 con 0x7f9f5013cee0 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f55ab8640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9f50140200 0x7f9f5015a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38568/0 (socket says 172.21.15.111:38568) 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f55ab8640 1 -- 172.21.15.111:0/1564005415 learned_addr learned my addr 172.21.15.111:0/1564005415 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f56aba640 1 --2- 172.21.15.111:0/1564005415 >> v2:172.21.15.192:3300/0 conn(0x7f9f5015aba0 0x7f9f5006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f562b9640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 0x7f9f5006c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281833796 0 0) 0x7f9f500aea20 con 0x7f9f50140200 2023-12-04T05:46:37.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f3c003520 con 0x7f9f50140200 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168726178 0 0) 0x7f9f3c003520 con 0x7f9f50140200 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 msgr2=0x7f9f5006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 0x7f9f5006c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 >> v2:172.21.15.192:3300/0 conn(0x7f9f5015aba0 msgr2=0x7f9f5006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 --2- 172.21.15.111:0/1564005415 >> v2:172.21.15.192:3300/0 conn(0x7f9f5015aba0 0x7f9f5006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5015b0c0 con 0x7f9f50140200 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1473980256 0 0) 0x7f9f40001670 con 0x7f9f50140200 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.479+0000 7f9f56aba640 1 --2- 172.21.15.111:0/1564005415 >> v2:172.21.15.192:3300/0 conn(0x7f9f5015aba0 0x7f9f5006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 >> v1:172.21.15.111:6789/0 conn(0x7f9f50140200 legacy=0x7f9f5015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 0x7f9f3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f3c004d00 0x7f9f3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5013d2c0 con 0x7f9f3c003970 2023-12-04T05:46:37.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015b3a0 con 0x7f9f3c004d00 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c00b980 con 0x7f9f5013cee0 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f562b9640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f55ab8640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 0x7f9f3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f56aba640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f3c004d00 0x7f9f3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f562b9640 1 -- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 msgr2=0x7f9f3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f562b9640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5013cee0 0x7f9f3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f562b9640 1 -- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f3c004d00 msgr2=0x7f9f3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f562b9640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f3c004d00 0x7f9f3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f562b9640 1 -- 172.21.15.111:0/1564005415 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c003520 con 0x7f9f3c003970 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.480+0000 7f9f56aba640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f3c004d00 0x7f9f3c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f562b9640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f3c003d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009aa0 tx=0x7f9f3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c9fa15b55b9d96 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c02c6b0 con 0x7f9f3c003970 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c02c810 con 0x7f9f3c003970 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f552b7640 1 -- 172.21.15.111:0/1564005415 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c02cbc0 con 0x7f9f3c003970 2023-12-04T05:46:37.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f5851d640 1 -- 172.21.15.111:0/1564005415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 msgr2=0x7f9f3c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f5851d640 1 --2- 172.21.15.111:0/1564005415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f3c003d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009aa0 tx=0x7f9f3c0057f0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f56aba640 1 -- 172.21.15.111:0/1564005415 reap_dead start 2023-12-04T05:46:37.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f5851d640 1 -- 172.21.15.111:0/1564005415 shutdown_connections 2023-12-04T05:46:37.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f5851d640 1 -- 172.21.15.111:0/1564005415 >> 172.21.15.111:0/1564005415 conn(0x7f9f5006a4c0 msgr2=0x7f9f5006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f5851d640 1 -- 172.21.15.111:0/1564005415 shutdown_connections 2023-12-04T05:46:37.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.481+0000 7f9f5851d640 1 -- 172.21.15.111:0/1564005415 wait complete. 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 Processor -- start 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 -- start start 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f501efe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f500add10 0x7f9f50139410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f50139950 0x7f9f500b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500b2640 con 0x7f9f3c003970 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500b2920 con 0x7f9f500add10 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f5851d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500b2c00 con 0x7f9f50139950 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f562b9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f501efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f562b9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f501efe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46126/0 (socket says 172.21.15.111:46126) 2023-12-04T05:46:37.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.483+0000 7f9f562b9640 1 -- 172.21.15.111:0/2109300797 learned_addr learned my addr 172.21.15.111:0/2109300797 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f55ab8640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f500add10 0x7f9f50139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f56aba640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f50139950 0x7f9f500b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f562b9640 1 -- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f50139950 msgr2=0x7f9f500b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f562b9640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f50139950 0x7f9f500b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f562b9640 1 -- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f500add10 msgr2=0x7f9f50139410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f562b9640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f500add10 0x7f9f50139410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f562b9640 1 -- 172.21.15.111:0/2109300797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500b2ee0 con 0x7f9f3c003970 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f562b9640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f501efe40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c037040 tx=0x7f9f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c040070 con 0x7f9f3c003970 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c034770 con 0x7f9f3c003970 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c034b00 con 0x7f9f3c003970 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f500b31c0 con 0x7f9f3c003970 2023-12-04T05:46:37.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.484+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500b2280 con 0x7f9f3c003970 2023-12-04T05:46:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.485+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f500693d0 con 0x7f9f3c003970 2023-12-04T05:46:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.485+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7f9f4c044070 con 0x7f9f3c003970 2023-12-04T05:46:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.486+0000 7f9f46ffd640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f3406b2d0 0x7f9f3406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.486+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f9f4c0aced0 con 0x7f9f3c003970 2023-12-04T05:46:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.486+0000 7f9f55ab8640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f3406b2d0 0x7f9f3406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.486+0000 7f9f55ab8640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f3406b2d0 0x7f9f3406d790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f501f3370 tx=0x7f9f40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.490+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f4c077c50 con 0x7f9f3c003970 2023-12-04T05:46:37.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.662+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f5006d170 con 0x7f9f3c003970 2023-12-04T05:46:37.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.663+0000 7f9f46ffd640 1 -- 172.21.15.111:0/2109300797 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16839 (secure 0 0 0) 0x7f9f4c039070 con 0x7f9f3c003970 2023-12-04T05:46:37.665 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:37.665 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":49,"fsid":"9686316e-a01b-4981-b0b7-e21bdb185d5e","created":"2023-12-04T05:38:52.422906+0000","modified":"2023-12-04T05:42:02.218551+0000","last_up_change":"2023-12-04T05:39:03.555002+0000","last_in_change":"2023-12-04T05:38:54.612762+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-04T05:39:04.516545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-04T05:41:00.843728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-04T05:41:01.612301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"130f386c-483a-4a82-9a0e-1a94f399374e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6801","nonce":378032548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6803","nonce":378032548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6807","nonce":378032548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":378032548},{"type":"v1","addr":"172.21.15.111:6805","nonce":378032548}]},"public_addr":"172.21.15.111:6801/378032548","cluster_addr":"172.21.15.111:6803/378032548","heartbeat_back_addr":"172.21.15.111:6807/378032548","heartbeat_front_addr":"172.21.15.111:6805/378032548","state":["exists","up"]},{"osd":1,"uuid":"5c1fd699-00ab-4e49-bfd2-d2d3b9406819","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6809","nonce":909770062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6811","nonce":909770062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6815","nonce":909770062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":909770062},{"type":"v1","addr":"172.21.15.111:6813","nonce":909770062}]},"public_addr":"172.21.15.111:6809/909770062","cluster_addr":"172.21.15.111:6811/909770062","heartbeat_back_addr":"172.21.15.111:6815/909770062","heartbeat_front_addr":"172.21.15.111:6813/909770062","state":["exists","up"]},{"osd":2,"uuid":"f801a2ee-b5d9-4d07-990b-2a19c10b639e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6819","nonce":306971668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6823","nonce":306971668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6830","nonce":306971668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":306971668},{"type":"v1","addr":"172.21.15.111:6827","nonce":306971668}]},"public_addr":"172.21.15.111:6819/306971668","cluster_addr":"172.21.15.111:6823/306971668","heartbeat_back_addr":"172.21.15.111:6830/306971668","heartbeat_front_addr":"172.21.15.111:6827/306971668","state":["exists","up"]},{"osd":3,"uuid":"6ce755a0-a57a-43e4-ae57-a640139bc901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6818","nonce":413639041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6822","nonce":413639041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6831","nonce":413639041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":413639041},{"type":"v1","addr":"172.21.15.111:6826","nonce":413639041}]},"public_addr":"172.21.15.111:6818/413639041","cluster_addr":"172.21.15.111:6822/413639041","heartbeat_back_addr":"172.21.15.111:6831/413639041","heartbeat_front_addr":"172.21.15.111:6826/413639041","state":["exists","up"]},{"osd":4,"uuid":"ab580f43-28c3-4b07-85a3-9140109388c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6806","nonce":465617341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6810","nonce":465617341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6815","nonce":465617341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":465617341},{"type":"v1","addr":"172.21.15.192:6813","nonce":465617341}]},"public_addr":"172.21.15.192:6806/465617341","cluster_addr":"172.21.15.192:6810/465617341","heartbeat_back_addr":"172.21.15.192:6815/465617341","heartbeat_front_addr":"172.21.15.192:6813/465617341","state":["exists","up"]},{"osd":5,"uuid":"70358cad-93c5-4a17-ac66-6a33bf19c0ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6801","nonce":659116358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6803","nonce":659116358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6811","nonce":659116358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":659116358},{"type":"v1","addr":"172.21.15.192:6807","nonce":659116358}]},"public_addr":"172.21.15.192:6801/659116358","cluster_addr":"172.21.15.192:6803/659116358","heartbeat_back_addr":"172.21.15.192:6811/659116358","heartbeat_front_addr":"172.21.15.192:6807/659116358","state":["exists","up"]},{"osd":6,"uuid":"db4a5ec8-5bf0-4980-8569-32f1e4e11d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6817","nonce":363847578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6822","nonce":363847578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6829","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6831","nonce":363847578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":363847578},{"type":"v1","addr":"172.21.15.192:6827","nonce":363847578}]},"public_addr":"172.21.15.192:6817/363847578","cluster_addr":"172.21.15.192:6822/363847578","heartbeat_back_addr":"172.21.15.192:6831/363847578","heartbeat_front_addr":"172.21.15.192:6827/363847578","state":["exists","up"]},{"osd":7,"uuid":"2bb896c5-8f9b-4a45-aa8d-84e0dd27e5f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6820","nonce":1243890085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6821","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6823","nonce":1243890085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6830","nonce":1243890085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6825","nonce":1243890085},{"type":"v1","addr":"172.21.15.192:6826","nonce":1243890085}]},"public_addr":"172.21.15.192:6820/1243890085","cluster_addr":"172.21.15.192:6823/1243890085","heartbeat_back_addr":"172.21.15.192:6830/1243890085","heartbeat_front_addr":"172.21.15.192:6826/1243890085","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.745246+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.750444+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.777128+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.815020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.774158+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.852451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.807083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-04T05:38:59.838500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6835/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.192:6833/956157809":"2023-12-05T05:42:02.218443+0000","172.21.15.111:6836/619465105":"2023-12-05T05:41:42.214046+0000","172.21.15.111:6834/619465105":"2023-12-05T05:41:42.214046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-04T05:46:37.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f3406b2d0 msgr2=0x7f9f3406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f3406b2d0 0x7f9f3406d790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f501f3370 tx=0x7f9f40009040 comp rx=0 tx=0).stop 2023-12-04T05:46:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 msgr2=0x7f9f501efe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f501efe40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c037040 tx=0x7f9f4c004730 comp rx=0 tx=0).stop 2023-12-04T05:46:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 shutdown_connections 2023-12-04T05:46:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7f9f3406b2d0 0x7f9f3406d790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f50139950 0x7f9f500b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f500add10 0x7f9f50139410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 --2- 172.21.15.111:0/2109300797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f3c003970 0x7f9f501efe40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 >> 172.21.15.111:0/2109300797 conn(0x7f9f5006a4c0 msgr2=0x7f9f5013d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 shutdown_connections 2023-12-04T05:46:37.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.665+0000 7f9f5851d640 1 -- 172.21.15.111:0/2109300797 wait complete. 2023-12-04T05:46:37.678 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-04T05:46:37.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:46:37.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.776+0000 7fdb8d47c640 1 Processor -- start 2023-12-04T05:46:37.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.776+0000 7fdb8d47c640 1 -- start start 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb8d47c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdb8815d4a0 0x7fdb880d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb8d47c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdb880d3850 0x7fdb880b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb8d47c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb880d27e0 con 0x7fdb880d2c70 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb8d47c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb880d1bd0 con 0x7fdb8815d4a0 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb8d47c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb880d1eb0 con 0x7fdb880d3850 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdb8815d4a0 0x7fdb880d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdb8815d4a0 0x7fdb880d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46140/0 (socket says 172.21.15.111:46140) 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 -- 172.21.15.111:0/4083989408 learned_addr learned my addr 172.21.15.111:0/4083989408 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb877fe640 1 --2- 172.21.15.111:0/4083989408 >> v2:172.21.15.192:3300/0 conn(0x7fdb880d3850 0x7fdb880b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 -- 172.21.15.111:0/4083989408 >> v2:172.21.15.192:3300/0 conn(0x7fdb880d3850 msgr2=0x7fdb880b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 --2- 172.21.15.111:0/4083989408 >> v2:172.21.15.192:3300/0 conn(0x7fdb880d3850 0x7fdb880b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 -- 172.21.15.111:0/4083989408 >> v1:172.21.15.192:6789/0 conn(0x7fdb880d2c70 legacy=0x7fdb880d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb86ffd640 1 -- 172.21.15.111:0/4083989408 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb880d2190 con 0x7fdb8815d4a0 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.777+0000 7fdb877fe640 1 --2- 172.21.15.111:0/4083989408 >> v2:172.21.15.192:3300/0 conn(0x7fdb880d3850 0x7fdb880b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.778+0000 7fdb86ffd640 1 --2- 172.21.15.111:0/4083989408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8815d4a0 0x7fdb880d12a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdb880b5c80 tx=0x7fdb7802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a223d8903630fd21 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.778+0000 7fdb867fc640 1 -- 172.21.15.111:0/4083989408 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78031870 con 0x7fdb8815d4a0 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.778+0000 7fdb867fc640 1 -- 172.21.15.111:0/4083989408 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb780319d0 con 0x7fdb8815d4a0 2023-12-04T05:46:37.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.778+0000 7fdb867fc640 1 -- 172.21.15.111:0/4083989408 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78031d60 con 0x7fdb8815d4a0 2023-12-04T05:46:37.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 -- 172.21.15.111:0/4083989408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8815d4a0 msgr2=0x7fdb880d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/4083989408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8815d4a0 0x7fdb880d12a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdb880b5c80 tx=0x7fdb7802b910 comp rx=0 tx=0).stop 2023-12-04T05:46:37.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 -- 172.21.15.111:0/4083989408 shutdown_connections 2023-12-04T05:46:37.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/4083989408 >> v2:172.21.15.192:3300/0 conn(0x7fdb880d3850 0x7fdb880b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/4083989408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8815d4a0 0x7fdb880d12a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 -- 172.21.15.111:0/4083989408 >> 172.21.15.111:0/4083989408 conn(0x7fdb88072010 msgr2=0x7fdb88072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 -- 172.21.15.111:0/4083989408 shutdown_connections 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.779+0000 7fdb8d47c640 1 -- 172.21.15.111:0/4083989408 wait complete. 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.780+0000 7fdb8d47c640 1 Processor -- start 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.780+0000 7fdb8d47c640 1 -- start start 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb8d47c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 0x7fdb880caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb8d47c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb880d3850 0x7fdb880c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb8d47c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb8815d4a0 0x7fdb8815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb8d47c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880c71c0 con 0x7fdb880d2c70 2023-12-04T05:46:37.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb8d47c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8815ee10 con 0x7fdb880d3850 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb8d47c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8815f0f0 con 0x7fdb8815d4a0 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 0x7fdb880caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 0x7fdb880caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46154/0 (socket says 172.21.15.111:46154) 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 -- 172.21.15.111:0/871189333 learned_addr learned my addr 172.21.15.111:0/871189333 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb877fe640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb8815d4a0 0x7fdb8815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb7ffff640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb880d3850 0x7fdb880c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 -- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb8815d4a0 msgr2=0x7fdb8815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb8815d4a0 0x7fdb8815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 -- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb880d3850 msgr2=0x7fdb880c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb880d3850 0x7fdb880c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 -- 172.21.15.111:0/871189333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb780094d0 con 0x7fdb880d2c70 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb877fe640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb8815d4a0 0x7fdb8815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:37.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.781+0000 7fdb86ffd640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 0x7fdb880caeb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdb7802b950 tx=0x7fdb780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.782+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7803ad30 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.782+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb780443f0 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.782+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780447a0 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.782+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb880cba00 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.782+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb880cbfc0 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.783+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb880693d0 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.783+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7fdb7804d450 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.783+0000 7fdb7f7fe640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb6806b2a0 0x7fdb6806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.783+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fdb7802d040 con 0x7fdb880d2c70 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.783+0000 7fdb7ffff640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb6806b2a0 0x7fdb6806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:37.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.784+0000 7fdb7ffff640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb6806b2a0 0x7fdb6806d760 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb880b8830 tx=0x7fdb70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:37.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.787+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb78077020 con 0x7fdb880d2c70 2023-12-04T05:46:37.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.987+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb880b3220 con 0x7fdb880d2c70 2023-12-04T05:46:37.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.987+0000 7fdb7f7fe640 1 -- 172.21.15.111:0/871189333 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3400 (secure 0 0 0) 0x7fdb78035d60 con 0x7fdb880d2c70 2023-12-04T05:46:37.989 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-04T05:46:37.989 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-04T05:41:02.644246+0000","modified":"2023-12-04T05:45:11.358964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5868,"mds_1":5042},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5042":{"gid":5042,"name":"b","rank":1,"incarnation":30,"state":"up:active","state_seq":20,"addr":"172.21.15.192:6834/773206989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":773206989},{"type":"v1","addr":"172.21.15.192:6834","nonce":773206989}]},"join_fscid":-1,"laggy_since":"2023-12-04T05:42:22.222606+0000","export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5868":{"gid":5868,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":65,"addr":"172.21.15.111:6837/2442390644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6835","nonce":2442390644},{"type":"v1","addr":"172.21.15.111:6837","nonce":2442390644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-04T05:46:37.989 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 37 2023-12-04T05:46:37.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb6806b2a0 msgr2=0x7fdb6806d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb6806b2a0 0x7fdb6806d760 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb880b8830 tx=0x7fdb70009040 comp rx=0 tx=0).stop 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 msgr2=0x7fdb880caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 0x7fdb880caeb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdb7802b950 tx=0x7fdb780046c0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 shutdown_connections 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7fdb6806b2a0 0x7fdb6806d760 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb8815d4a0 0x7fdb8815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb880d3850 0x7fdb880c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 --2- 172.21.15.111:0/871189333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb880d2c70 0x7fdb880caeb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 >> 172.21.15.111:0/871189333 conn(0x7fdb88072010 msgr2=0x7fdb880d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 shutdown_connections 2023-12-04T05:46:37.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:37.989+0000 7fdb8d47c640 1 -- 172.21.15.111:0/871189333 wait complete. 2023-12-04T05:46:38.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-04T05:46:38.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.101+0000 7fafa56ee640 1 Processor -- start 2023-12-04T05:46:38.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.101+0000 7fafa56ee640 1 -- start start 2023-12-04T05:46:38.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7fafa56ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafa015d4a0 0x7fafa00d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7fafa56ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa00d27e0 con 0x7fafa00d3850 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7fafa56ee640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa00d1bd0 con 0x7fafa00d2c70 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7fafa56ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa00d1eb0 con 0x7fafa015d4a0 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9effd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafa015d4a0 0x7fafa00d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fafa00d3850 0x7fafa00b62e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38578/0 (socket says 172.21.15.111:38578) 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9effd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fafa015d4a0 0x7fafa00d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46160/0 (socket says 172.21.15.111:46160) 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9f7fe640 1 -- 172.21.15.111:0/3501338414 learned_addr learned my addr 172.21.15.111:0/3501338414 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9effd640 1 -- 172.21.15.111:0/3501338414 >> v1:172.21.15.192:6789/0 conn(0x7fafa00d2c70 legacy=0x7fafa00d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9effd640 1 -- 172.21.15.111:0/3501338414 >> v1:172.21.15.111:6789/0 conn(0x7fafa00d3850 legacy=0x7fafa00b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9effd640 1 -- 172.21.15.111:0/3501338414 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa00d2190 con 0x7fafa015d4a0 2023-12-04T05:46:38.104 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.102+0000 7faf9e7fc640 1 -- 172.21.15.111:0/3501338414 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221572338 0 0) 0x7fafa00d27e0 con 0x7fafa00d3850 2023-12-04T05:46:38.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7faf9effd640 1 --2- 172.21.15.111:0/3501338414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa00d12a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faf900097a0 tx=0x7faf9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b403407532ef884e server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:38.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7faf9e7fc640 1 -- 172.21.15.111:0/3501338414 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9002e070 con 0x7fafa015d4a0 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7faf9e7fc640 1 -- 172.21.15.111:0/3501338414 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf900315a0 con 0x7fafa015d4a0 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7faf9e7fc640 1 -- 172.21.15.111:0/3501338414 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90031930 con 0x7fafa015d4a0 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7fafa56ee640 1 -- 172.21.15.111:0/3501338414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 msgr2=0x7fafa00d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7fafa56ee640 1 --2- 172.21.15.111:0/3501338414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa00d12a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faf900097a0 tx=0x7faf9002b840 comp rx=0 tx=0).stop 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7fafa56ee640 1 -- 172.21.15.111:0/3501338414 shutdown_connections 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7fafa56ee640 1 --2- 172.21.15.111:0/3501338414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa00d12a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.103+0000 7fafa56ee640 1 -- 172.21.15.111:0/3501338414 >> 172.21.15.111:0/3501338414 conn(0x7fafa0072010 msgr2=0x7fafa0072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.104+0000 7fafa56ee640 1 -- 172.21.15.111:0/3501338414 shutdown_connections 2023-12-04T05:46:38.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.104+0000 7fafa56ee640 1 -- 172.21.15.111:0/3501338414 wait complete. 2023-12-04T05:46:38.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 Processor -- start 2023-12-04T05:46:38.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 -- start start 2023-12-04T05:46:38.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7fafa015ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7fafa00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa00cc4a0 con 0x7fafa015d4a0 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa00cc010 con 0x7fafa00d2c70 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7fafa56ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa00cb170 con 0x7fafa00d3850 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7faf9f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7faf9f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46170/0 (socket says 172.21.15.111:46170) 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.105+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 learned_addr learned my addr 172.21.15.111:0/718615653 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7fafa015ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7fafa00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7fafa00cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7fafa015ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7fafa015ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:38.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90009400 con 0x7fafa015d4a0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7fafa00cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c780 tx=0x7faf9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800ce90 con 0x7fafa015d4a0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9800a640 con 0x7fafa015d4a0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7fafa56ee640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa00cb400 con 0x7fafa015d4a0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800aa30 con 0x7fafa015d4a0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.106+0000 7fafa56ee640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa00cb9c0 con 0x7fafa015d4a0 2023-12-04T05:46:38.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.107+0000 7fafa56ee640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa00cc1c0 con 0x7fafa015d4a0 2023-12-04T05:46:38.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.107+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 89549+0+0 (secure 0 0 0) 0x7faf9800ab90 con 0x7fafa015d4a0 2023-12-04T05:46:38.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.108+0000 7faf977fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7faf8006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.108+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7faf8006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-04T05:46:38.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.108+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5939+0+0 (secure 0 0 0) 0x7faf98089140 con 0x7fafa015d4a0 2023-12-04T05:46:38.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.108+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7faf8006d760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf90036000 tx=0x7faf90008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-04T05:46:38.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.112+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fafa00cc1c0 con 0x7fafa015d4a0 2023-12-04T05:46:38.184 INFO:tasks.ceph.osd.0:Sent signal 15 2023-12-04T05:46:38.185 INFO:tasks.ceph.osd.1:Sent signal 15 2023-12-04T05:46:38.185 INFO:tasks.ceph.osd.2:Sent signal 15 2023-12-04T05:46:38.185 INFO:tasks.ceph.osd.3:Sent signal 15 2023-12-04T05:46:38.185 INFO:tasks.ceph.osd.4:Sent signal 15 2023-12-04T05:46:38.186 INFO:tasks.ceph.osd.5:Sent signal 15 2023-12-04T05:46:38.186 INFO:tasks.ceph.osd.6:Sent signal 15 2023-12-04T05:46:38.186 INFO:tasks.ceph.osd.7:Sent signal 15 2023-12-04T05:46:38.186 INFO:tasks.ceph.mds.c:Sent signal 15 2023-12-04T05:46:38.186 INFO:tasks.ceph.mon.a:Sent signal 15 2023-12-04T05:46:38.187 INFO:tasks.ceph.mon.b:Sent signal 15 2023-12-04T05:46:38.187 INFO:tasks.ceph.mon.c:Sent signal 15 2023-12-04T05:46:38.187 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-12-04T05:46:38.187 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-12-04T05:46:38.188 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:46:38.185+0000 7fd886a2e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 89790) UID: 0 2023-12-04T05:46:38.188 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:46:38.185+0000 7fd886a2e640 -1 osd.4 49 *** Got signal Terminated *** 2023-12-04T05:46:38.188 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-04T05:46:38.185+0000 7fd886a2e640 -1 osd.4 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.188 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f15b7435640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 88952) UID: 0 2023-12-04T05:46:38.188 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f15b7435640 -1 osd.0 49 *** Got signal Terminated *** 2023-12-04T05:46:38.188 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f15b7435640 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.189 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:46:38.185+0000 7f2b552fa640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 89788) UID: 0 2023-12-04T05:46:38.189 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:46:38.185+0000 7f2b552fa640 -1 osd.5 49 *** Got signal Terminated *** 2023-12-04T05:46:38.189 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-04T05:46:38.185+0000 7f2b552fa640 -1 osd.5 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.189 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f8520219640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 88956) UID: 0 2023-12-04T05:46:38.189 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f8520219640 -1 osd.1 49 *** Got signal Terminated *** 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f8520219640 -1 osd.1 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:46:38.185+0000 7f71bfbbd640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 89792) UID: 0 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:46:38.185+0000 7f71bfbbd640 -1 osd.6 49 *** Got signal Terminated *** 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-04T05:46:38.185+0000 7f71bfbbd640 -1 osd.6 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f7d49227640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 88955) UID: 0 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f7d49227640 -1 osd.2 49 *** Got signal Terminated *** 2023-12-04T05:46:38.190 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-12-04T05:46:38.184+0000 7f7d49227640 -1 osd.2 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.191 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:46:38.186+0000 7fd9e37dc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 89794) UID: 0 2023-12-04T05:46:38.191 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:46:38.186+0000 7fd9e37dc640 -1 osd.7 49 *** Got signal Terminated *** 2023-12-04T05:46:38.191 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-04T05:46:38.186+0000 7fd9e37dc640 -1 osd.7 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.191 INFO:tasks.ceph.mon.b.smithi192.stderr:2023-12-04T05:46:38.186+0000 7f809f390640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 89219) UID: 0 2023-12-04T05:46:38.191 INFO:tasks.ceph.mon.b.smithi192.stderr:2023-12-04T05:46:38.186+0000 7f809f390640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-12-04T05:46:38.191 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:46:38.185+0000 7fb4131e1640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 88959) UID: 0 2023-12-04T05:46:38.192 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:46:38.185+0000 7fb4131e1640 -1 osd.3 49 *** Got signal Terminated *** 2023-12-04T05:46:38.192 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-12-04T05:46:38.185+0000 7fb4131e1640 -1 osd.3 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-04T05:46:38.192 INFO:tasks.ceph.mon.c.smithi192.stderr:2023-12-04T05:46:38.187+0000 7fdaa1128640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 89221) UID: 0 2023-12-04T05:46:38.192 INFO:tasks.ceph.mon.c.smithi192.stderr:2023-12-04T05:46:38.187+0000 7fdaa1128640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-12-04T05:46:38.192 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:46:38.186+0000 7f344a465640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 107131) UID: 0 2023-12-04T05:46:38.192 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-12-04T05:46:38.186+0000 7f344a465640 -1 mds.c *** got signal Terminated *** 2023-12-04T05:46:38.192 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-12-04T05:46:38.186+0000 7fb28290f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 88688) UID: 0 2023-12-04T05:46:38.193 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-12-04T05:46:38.186+0000 7fb28290f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-04T05:46:38.193 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:46:38.186+0000 7f8687482640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i y (PID: 88725) UID: 0 2023-12-04T05:46:38.193 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-12-04T05:46:38.186+0000 7f8687482640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 msgr2=0x7fafa01f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 msgr2=0x7fafa01f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c780 tx=0x7faf9800cc40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c780 tx=0x7faf9800cc40 comp rx=0 tx=0).stop 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 msgr2=0x7fafa01f2d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafa015d4a0 0x7fafa01f2d40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf80075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf80077d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf8007ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7fafa00d2c70 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf80075540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf80075540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.188+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf80077d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.189+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf80077d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.189+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf8007ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.189+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf8007ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:38.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.205+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7faf8006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2023-12-04T05:46:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.205+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7faf8006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-04T05:46:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.205+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7faf8006d760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf90036000 tx=0x7faf90008030 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-04T05:46:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.205+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7faf8006d760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf90036000 tx=0x7faf90008030 comp rx=0 tx=0).stop 2023-12-04T05:46:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.205+0000 7faf977fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7faf8006d760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-12-04T05:46:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.205+0000 7faf977fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7faf8006d760 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:38.389 INFO:tasks.ceph.mgr.x.smithi192.stderr:daemon-helper: command crashed with signal 15 2023-12-04T05:46:38.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.389+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf80075540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:38.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.389+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf80075540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:38.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.390+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf8007ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:38.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.390+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf8007ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:38.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.390+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf80077d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:38.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.390+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf80077d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:38.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.790+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf80075540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:38.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.790+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf80075540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:38.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.791+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf8007ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:38.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.791+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf8007ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:38.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.791+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf80077d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:38.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:38.791+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf80077d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:39.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.108+0000 7faf95ffb640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:39.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.108+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:39.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.108+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:39.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.309+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:39.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.309+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.591+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf80075540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.591+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf80075540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.592+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf8007ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.592+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf8007ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.592+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf80077d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.592+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf80077d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:39.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.710+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:39.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:39.710+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:40.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:40.511+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:40.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:40.511+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf80077d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf80077d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf80075540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf80075540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf8007ab30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf8007ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7fafa00d2c70 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.106+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:41.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.307+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:41.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.307+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:41.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.307+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:41.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.307+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:41.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.307+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:41.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.307+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:41.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.708+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:41.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.708+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:41.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.708+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:41.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.708+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:41.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.708+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:41.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:41.708+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:42.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.112+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:42.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.112+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:42.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.509+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:42.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.509+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:42.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.509+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:42.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.509+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:42.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.509+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:42.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:42.509+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:44.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:44.111+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:44.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:44.111+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:44.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:44.111+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:44.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:44.111+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:44.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:44.111+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:44.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:44.111+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:45.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:45.313+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:45.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:45.313+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:47.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.313+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:47.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.313+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:47.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.314+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:47.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.314+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:47.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.314+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:47.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.314+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:47.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c007c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:47.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c007c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 msgr2=0x7faf7c00a350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa00d2c70 0x7faf7c00a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c005620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c005620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:47.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7faf7c00ad10 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:47.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:47.856+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:48.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.057+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:48.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.057+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:48.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.057+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:48.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.057+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:48.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.057+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:48.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.057+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:48.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:46:48.232 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:46:48.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.458+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:48.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.458+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:48.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.458+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:48.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.458+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:48.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.458+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:48.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:48.458+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:49.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:49.259+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:49.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:49.259+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:49.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:49.259+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:49.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:49.259+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:49.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:49.259+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:49.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:49.259+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:50.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:50.861+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:50.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:50.861+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:50.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:50.861+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:50.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:50.861+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:50.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:50.861+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:50.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:50.861+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:46:51.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:51.714+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:46:51.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:51.714+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:46:54.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:54.063+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:54.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:54.063+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:54.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:54.064+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:54.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:54.064+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:54.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:54.064+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:54.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:54.064+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:46:57.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00d100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7faf7c00ad10 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:46:57.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:57.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:57.982+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:46:58.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.183+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:58.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.183+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:58.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.183+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:58.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.183+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:58.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.183+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:58.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.183+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:46:58.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.584+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:58.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.584+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:58.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.584+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:58.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.584+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:58.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.584+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:58.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:58.584+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:46:59.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:59.385+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:46:59.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:59.385+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:59.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:59.385+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:46:59.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:59.385+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:46:59.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:59.385+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:46:59.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:46:59.385+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:00.986+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:00.986+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:00.986+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:00.986+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:00.986+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:00.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:00.986+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:04.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.190+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:04.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.190+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:04.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.190+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:04.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.190+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:04.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.190+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:04.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.190+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:04.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.515+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:47:04.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:04.515+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:47:10.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:10.595+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:10.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:10.595+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:10.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:10.596+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:10.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:10.596+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:10.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:10.596+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:10.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:10.596+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:47:13.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7faf7c00ad10 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:13.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.170+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:47:13.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.371+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:13.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.371+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:47:13.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.371+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:13.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.371+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:47:13.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.371+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:13.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.371+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:47:13.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.772+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:13.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.772+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:47:13.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.772+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:13.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.772+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:13.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.772+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:47:13.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:13.772+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:47:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:14.573+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:14.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:14.573+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:14.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:14.573+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:14.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:14.573+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:14.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:14.573+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:14.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:14.573+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:16.175+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:16.175+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:16.175+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:16.175+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:16.175+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:16.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:16.175+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:18.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:47:18.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:47:19.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.378+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:19.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.378+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:19.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.378+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:19.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.378+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:19.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.378+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:19.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.378+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:19.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.516+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:47:19.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:19.516+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:47:25.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:25.785+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:25.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:25.785+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:25.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:25.785+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:25.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:25.785+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:25.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:25.785+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:25.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:25.785+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:34.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:34.524+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:47:34.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:34.524+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:47:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7faf7c00ad10 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:47:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:35.951+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:47:36.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.152+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:36.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.152+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:47:36.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.152+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:36.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.152+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:36.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.152+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:47:36.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.152+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:47:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.553+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.553+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:47:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.553+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.553+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.553+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:47:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:36.553+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:47:37.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:37.354+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:37.354+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:37.354+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:37.354+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:37.354+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:37.354+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:47:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:38.955+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:38.955+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:38.956+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:38.956+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:38.956+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:38.956+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:47:42.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:42.159+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:42.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:42.159+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:42.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:42.159+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:42.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:42.159+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:42.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:42.159+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:42.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:42.159+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:47:48.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:48.566+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:47:48.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:48.566+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:48.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:48.566+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:47:48.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:48.566+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:48.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:48.566+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:47:48.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:48.566+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:47:48.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:47:48.614 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:47:49.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:49.525+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:47:49.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:47:49.525+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:01.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:01.378+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:01.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:01.378+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:01.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:01.379+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:01.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:01.379+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:01.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:01.379+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:01.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:01.379+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:04.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:04.529+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:48:04.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:04.529+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:05.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:48:05.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:05.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:48:05.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7faf7c00ad10 2023-12-04T05:48:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:48:05.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:05.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:05.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:05.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:05.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:05.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:05.952+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:06.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.153+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:06.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.153+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:06.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.153+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:06.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.153+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:06.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.153+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:06.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.153+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.554+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.554+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.554+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.554+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:06.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.554+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:06.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:06.554+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:07.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:07.355+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:07.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:07.355+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:07.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:07.355+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:07.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:07.355+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:07.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:07.355+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:07.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:07.355+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:08.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:08.957+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:08.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:08.957+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:48:08.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:08.957+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:08.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:08.957+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:08.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:08.957+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:48:08.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:08.957+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-04T05:48:12.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:12.160+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:12.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:12.160+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:48:12.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:12.160+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:12.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:12.160+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:12.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:12.160+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:48:12.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:12.160+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-04T05:48:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:18.567+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:18.567+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:48:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:18.567+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:18.567+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:18.567+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:48:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:18.567+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-12-04T05:48:18.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:48:18.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:48:19.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:19.530+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:48:19.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:19.530+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:31.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:31.379+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:31.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:31.379+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:31.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:31.379+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:31.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:31.379+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:31.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:31.379+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:31.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:31.379+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:34.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:34.532+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 msgr2=0x7fafa00cac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6832/3900317453 2023-12-04T05:48:34.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:34.532+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:6832/3900317453,v1:172.21.15.111:6833/3900317453] conn(0x7faf8006b2a0 0x7fafa00cac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-12-04T05:48:35.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:48:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:48:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-04T05:48:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:35.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b630 con 0x7faf800786a0 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b770 con 0x7faf7c00ad10 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf967fc640 1 -- 172.21.15.111:0/718615653 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8007b900 con 0x7fafa00d3850 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.951+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.952+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:35.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.952+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:35.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.952+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:35.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:35.952+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:36.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.152+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:36.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.152+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:36.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.153+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:36.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.153+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:36.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.153+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:36.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.153+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:36.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.553+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:36.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.553+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.554+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.554+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.554+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:36.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:36.554+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:37.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:37.354+0000 7faf97fff640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 msgr2=0x7faf7c003610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:37.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:37.354+0000 7faf97fff640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf800786a0 0x7faf7c003610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:37.355+0000 7faf9effd640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 msgr2=0x7faf7c008150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:37.355+0000 7faf9f7fe640 1 -- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 msgr2=0x7faf7c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:37.355+0000 7faf9effd640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa00d3850 0x7faf7c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:37.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:37.355+0000 7faf9f7fe640 1 --2- 172.21.15.111:0/718615653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf7c00ad10 0x7faf7c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:38.034 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-12-04T05:48:38.046 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client ... ERROR 2023-12-04T05:48:38.048 INFO:tasks.cephfs_test_runner:ERROR 2023-12-04T05:48:38.048 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.048 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:ERROR: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/test_sessionmap.py", line 213, in test_session_evict_blocklisted 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: self.mount_a.run_shell(["touch", "d1/f0"]) 2023-12-04T05:48:38.049 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: r.wait() 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 148, in wait 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 323, in copy_file_to 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 276, in copy_to_log 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: for line in f: 2023-12-04T05:48:38.050 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: line = self.readline() 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner:socket.timeout 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner:ERROR: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/cephfs_test_case.py", line 208, in tearDown 2023-12-04T05:48:38.051 INFO:tasks.cephfs_test_runner: self.mds_cluster.delete_all_filesystems() 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 494, in delete_all_filesystems 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: Filesystem(ctx=self._ctx, fscid=fs['id']).destroy() 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 840, in destroy 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: data_pools = self.get_data_pool_names(refresh=True) 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 970, in get_data_pool_names 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: self.get_pool_names(refresh = True) 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 939, in get_pool_names 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: status = self.status() 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-12-04T05:48:38.052 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: r.wait() 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner:Ran 2 tests in 535.978s 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-04T05:48:38.053 INFO:tasks.cephfs_test_runner:ERROR: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/test_sessionmap.py", line 213, in test_session_evict_blocklisted 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: self.mount_a.run_shell(["touch", "d1/f0"]) 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-12-04T05:48:38.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: r.wait() 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 148, in wait 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 323, in copy_file_to 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 276, in copy_to_log 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: for line in f: 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: line = self.readline() 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline 2023-12-04T05:48:38.055 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner:socket.timeout 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner:ERROR: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/cephfs_test_case.py", line 208, in tearDown 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: self.mds_cluster.delete_all_filesystems() 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 494, in delete_all_filesystems 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: Filesystem(ctx=self._ctx, fscid=fs['id']).destroy() 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 840, in destroy 2023-12-04T05:48:38.056 INFO:tasks.cephfs_test_runner: data_pools = self.get_data_pool_names(refresh=True) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 970, in get_data_pool_names 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: self.get_pool_names(refresh = True) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 939, in get_pool_names 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: status = self.status() 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: r.wait() 2023-12-04T05:48:38.057 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait 2023-12-04T05:48:38.058 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-12-04T05:48:38.058 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-12-04T05:48:38.058 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-12-04T05:48:38.058 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-12-04T05:48:38.058 INFO:tasks.cephfs_test_runner: 2023-12-04T05:48:38.058 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap), test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:48:38.137 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f33f8180c7ce4803923693d317b1b22e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap), test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2023-12-04T05:48:38.138 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-04T05:48:38.147 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-12-04T05:48:38.171 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-12-04T05:48:38.172 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:48:38.172 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:48:38.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:48:38.194 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-04T05:48:38.195 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:48:38.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056378,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17780 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-04T05:48:38.252 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:48:38.253 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:48:38.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:48:38.271 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-04T05:48:38.272 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-04T05:48:38.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056366,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248852k,nr_inodes=812213,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497708k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-04T05:48:38.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.329 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-12-04T05:48:38.346 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-04T05:48:38.347 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-04T05:48:38.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-04T05:48:38.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-04T05:48:38.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.490 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-12-04T05:48:38.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.588 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-04T05:48:38.605 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-04T05:48:38.606 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-04T05:48:38.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-04T05:48:38.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-04T05:48:38.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-04T05:48:38.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2023-12-04T05:48:38.843 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-04T05:48:38.852 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-04T05:48:38.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-04T05:48:38.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.954+0000 7f52da761640 1 Processor -- start 2023-12-04T05:48:38.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.954+0000 7f52da761640 1 -- start start 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52da761640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52d406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52da761640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52d40b3d20 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52da761640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52d4162490 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52da761640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52d40b3940 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52d406c250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52d40b3d20 legacy=0x7f52d4161d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:38.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52d406c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:38.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:38.955+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52d4162490 legacy=0x7f52d40703f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:39.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.156+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52d406c250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:39.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.156+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52d40b3d20 legacy=0x7f52d4161d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:39.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.156+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52d406c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:39.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.156+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52d4162490 legacy=0x7f52d40703f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:39.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.557+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52d40b3d20 legacy=0x7f52d4161d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:39.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.557+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52d406c250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:39.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.557+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52d406c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:39.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:39.557+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52d4162490 legacy=0x7f52d40703f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:40.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:40.358+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52d406c250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:40.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:40.358+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52d40b3d20 legacy=0x7f52d4161d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:40.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:40.358+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52d406c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:40.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:40.358+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52d4162490 legacy=0x7f52d40703f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:41.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52d406c250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:41.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52d406c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:41.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52d4162490 legacy=0x7f52d40703f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52d40b3d20 legacy=0x7f52d4161d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 0x7f52c00085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006130 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c00036a0 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 msgr2=0x7f52c00085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 0x7f52c00085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:41.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:41.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:41.955+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:42.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.156+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:42.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.156+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 msgr2=0x7f52c00085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:42.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.156+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:42.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.156+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 0x7f52c00085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:42.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.557+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:42.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.557+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 msgr2=0x7f52c00085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:42.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.557+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:42.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:42.557+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 0x7f52c00085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:43.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:43.358+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:43.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:43.358+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 msgr2=0x7f52c00085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:43.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:43.358+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:43.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:43.358+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 0x7f52c00085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:44.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:44.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:44.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 msgr2=0x7f52c00085f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0006130 0x7f52c00085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 0x7f52c000b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52c000da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52d40b3940 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c0008fe0 2023-12-04T05:48:44.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0006670 con 0x7f52c00036a0 2023-12-04T05:48:44.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52c000da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:44.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52c000da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:44.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 msgr2=0x7f52c000b3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:44.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:44.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 0x7f52c000b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:44.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:44.955+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:45.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.156+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52c000da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:45.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.156+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52c000da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:45.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.156+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:45.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.156+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 msgr2=0x7f52c000b3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:45.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.156+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 0x7f52c000b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:45.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.156+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:45.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.557+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52c000da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:45.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.557+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52c000da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:45.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.557+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 msgr2=0x7f52c000b3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:45.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.557+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:45.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.557+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 0x7f52c000b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:45.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:45.557+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:46.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:46.358+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52c000da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:46.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:46.358+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52c000da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:46.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:46.358+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 msgr2=0x7f52c000b3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:46.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:46.358+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:46.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:46.358+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 0x7f52c000b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:46.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:46.358+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:47.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:47.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00036a0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:47.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 msgr2=0x7f52c000b3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:47.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0008fe0 0x7f52c000b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:47.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 msgr2=0x7f52c000da50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:47.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52d40b3940 0x7f52c000da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:47.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c00085b0 2023-12-04T05:48:47.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c00068f0 2023-12-04T05:48:47.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00085b0 legacy=0x7f52c0011fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:47.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:47.956+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00068f0 legacy=0x7f52c0006d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:48.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:48.157+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00085b0 legacy=0x7f52c0011fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:48.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:48.157+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00068f0 legacy=0x7f52c0006d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:48.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:48.558+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00085b0 legacy=0x7f52c0011fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:48.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:48.558+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00068f0 legacy=0x7f52c0006d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:49.050 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:48:49.053 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:48:49.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:49.359+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00085b0 legacy=0x7f52c0011fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:49.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:49.359+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00068f0 legacy=0x7f52c0006d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:50.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00068f0 legacy=0x7f52c0006d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:50.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00085b0 legacy=0x7f52c0011fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:50.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:48:50.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c00074b0 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000b3d0 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:50.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:50.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:50.956+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:51.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.157+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:51.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.157+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:51.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.157+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:51.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.157+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:51.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.157+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:51.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.558+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:51.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.558+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:51.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.558+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:51.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.558+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:51.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:51.558+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:52.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:52.359+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:52.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:52.359+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:52.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:52.359+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:52.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:52.359+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:52.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:52.359+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:53.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:53.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:53.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:53.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 0x7f52c000ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c000d3e0 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000ca30 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 msgr2=0x7f52c000ce70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:53.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:53.956+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 0x7f52c000ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:54.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:54.157+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:54.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:54.157+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 msgr2=0x7f52c000ce70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:54.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:54.157+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 0x7f52c000ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:54.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:54.558+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:54.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:54.558+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 msgr2=0x7f52c000ce70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:54.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:54.558+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 0x7f52c000ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:55.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:55.359+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:55.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:55.359+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 msgr2=0x7f52c000ce70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:55.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:55.359+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 0x7f52c000ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:56.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 msgr2=0x7f52c000ce70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:56.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000ca30 0x7f52c000ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:56.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:48:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c00074b0 2023-12-04T05:48:56.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:48:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000b3d0 2023-12-04T05:48:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:56.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:56.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:56.956+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:48:57.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.157+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:57.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.157+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:57.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.157+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:57.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.157+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:57.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.157+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:48:57.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.558+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:57.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.558+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:57.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.558+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:57.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.558+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:57.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:57.558+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:48:58.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:58.359+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:48:58.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:58.359+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:48:58.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:58.359+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:58.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:58.359+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:48:58.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:58.359+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:48:59.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:59.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:59.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:59.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:48:59.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00074b0 legacy=0x7f52c0007890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:48:59.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c000ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:48:59.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c000d3e0 2023-12-04T05:48:59.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:48:59.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c000ca30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:48:59.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:48:59.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:48:59.957+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c000ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:00.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:00.158+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c000ca30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:00.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:00.158+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:00.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:00.158+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c000ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:00.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:00.559+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c000ca30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:00.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:00.559+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:00.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:00.559+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c000ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:01.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:01.360+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:01.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:01.360+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c000ca30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:01.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:01.360+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c000ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:02.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c000ca30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c000ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000d3e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c00151e0 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c0006d00 2023-12-04T05:49:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000b3d0 2023-12-04T05:49:02.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:02.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:02.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:02.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:02.957+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.158+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.158+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.158+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:03.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.158+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:03.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.559+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:03.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.559+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:03.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.559+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:03.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:03.559+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:04.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:04.360+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:04.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:04.360+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:04.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:04.360+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:04.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:04.360+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:05.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:05.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:05.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:05.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:05.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 0x7f52c0005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cb60 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c0007bc0 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:05.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:05.957+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:06.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.158+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:06.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.158+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:06.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.158+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:06.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.158+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:06.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.559+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:06.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.559+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:06.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.559+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:06.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:06.559+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:07.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:07.360+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:07.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:07.360+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:07.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:07.360+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:07.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:07.360+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:08.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 msgr2=0x7f52c0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:08.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0007bc0 0x7f52c0005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:08.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c0006d00 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cb60 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cfe0 2023-12-04T05:49:08.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:08.958+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:09.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.159+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:09.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.159+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:09.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.159+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:09.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.159+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:09.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.159+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:09.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.560+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:09.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.560+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:09.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.560+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:09.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.560+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:09.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:09.560+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:10.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:10.361+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:10.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:10.361+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:10.361+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:10.361+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:10.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:10.361+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:11.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c000b3d0 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:49:11.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:11.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:11.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:11.958+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:12.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:12.159+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:12.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:12.159+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:12.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:12.159+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:12.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:12.560+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:12.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:12.560+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:12.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:12.560+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:13.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:13.361+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:13.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:13.361+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:13.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:13.361+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:14.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:14.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:14.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:14.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:14.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:14.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c0006d00 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cb60 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c00151e0 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:14.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:14.958+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:15.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.159+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:15.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.159+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:15.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.159+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:15.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.159+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:15.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.159+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:15.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.560+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:15.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.560+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:15.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.560+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:15.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.560+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:15.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:15.560+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:16.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:16.361+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:16.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:16.361+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:16.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:16.361+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:16.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:16.361+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:16.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:16.361+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:17.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:17.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:17.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0015750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:17.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cb60 0x7f52c0015750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:17.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c0007410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:17.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0015c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:17.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c000b3d0 2023-12-04T05:49:17.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:17.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:17.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0015c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:17.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:17.959+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0015c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:18.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:18.160+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:18.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:18.160+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0015c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:18.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:18.160+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0015c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:18.561+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:18.561+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0015c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:18.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:18.561+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0015c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:19.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:49:19.138 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:49:19.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:19.362+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:19.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:19.362+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0015c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:19.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:19.362+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0015c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:20.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0015c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:20.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0015c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000b3d0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c00151e0 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0006d00 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cb60 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:20.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:20.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:20.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:20.959+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:21.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.160+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:21.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.160+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:21.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.160+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:21.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.160+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:21.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.561+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:21.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.561+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:21.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.561+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:21.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:21.561+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:22.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:22.362+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:22.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:22.362+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:22.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:22.362+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:22.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:22.362+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:23.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0017760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000b3d0 2023-12-04T05:49:23.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:23.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:23.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:23.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:23.958+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:24.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.159+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:24.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.159+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:24.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.159+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:24.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.159+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:24.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.560+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:24.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.560+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:24.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.560+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:24.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:24.560+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:25.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:25.361+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:25.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:25.361+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:25.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:25.361+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:25.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:25.361+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:26.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:26.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000b3d0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:26.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:26.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:26.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:26.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.958+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c0012f90 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0006670 con 0x7f52c000cb60 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:26.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:26.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:26.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:26.959+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:27.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.160+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:27.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.160+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:27.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.160+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:27.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.160+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:27.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.160+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:27.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.160+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:27.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.561+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:27.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.561+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:27.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.561+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:27.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.561+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:27.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.561+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:27.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:27.561+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:28.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:28.362+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:28.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:28.362+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:28.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:28.362+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:28.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:28.362+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:28.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:28.362+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:28.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:28.362+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0017760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0017760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c0006d00 2023-12-04T05:49:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c00151e0 2023-12-04T05:49:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:29.959+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:30.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:30.160+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:30.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:30.160+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:30.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:30.561+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:30.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:30.561+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:31.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:31.362+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:31.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:31.362+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c00138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c00151e0 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cb60 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c00138b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c00138b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:32.959+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:33.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.160+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c00138b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:33.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.160+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c00138b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:33.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.160+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:33.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.160+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:33.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.160+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:33.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.561+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c00138b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:33.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.561+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c00138b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:33.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.561+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:33.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.561+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:33.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:33.561+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:34.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:34.362+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c00138b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:34.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:34.362+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c00138b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:34.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:34.362+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:34.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:34.362+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:34.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:34.362+0000 7f52d8cfe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:35.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:35.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:35.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c00138b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:35.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c00138b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c0006d00 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0012f90 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:35.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:35.959+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:36.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:36.160+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:36.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:36.160+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:36.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:36.160+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:36.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:36.561+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:36.561+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:36.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:36.561+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:37.362+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:37.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:37.362+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:37.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:37.362+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:38.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c0013370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c0013370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0006d00 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c00151e0 2023-12-04T05:49:38.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.959+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cb60 2023-12-04T05:49:38.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.960+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:38.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.960+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:38.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.960+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:38.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:38.960+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:39.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.161+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:39.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.161+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:39.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.161+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:39.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.161+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:39.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.562+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:39.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.562+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:39.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.562+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:39.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:39.562+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:40.363+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:40.363+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:40.363+0000 7f52d3fff640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:40.363+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:41.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:41.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:41.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:41.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:41.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:41.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:41.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:41.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c00134d0 2023-12-04T05:49:41.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:41.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:41.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:41.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:41.960+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:42.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.161+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:42.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.161+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:42.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.161+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:42.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.161+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:42.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.562+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:42.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.562+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:42.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.562+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:42.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:42.562+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:43.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:43.363+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:43.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:43.363+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:43.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:43.363+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:43.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:43.363+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:44.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:44.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:44.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c00151e0 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:44.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c00134d0 2023-12-04T05:49:44.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:44.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:44.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:44.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:44.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:44.960+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:45.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.161+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:45.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.161+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:45.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.161+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:45.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.161+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:45.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.161+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:45.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.562+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:45.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.562+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:45.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.562+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:45.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.562+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:45.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:45.562+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:46.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:46.363+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:46.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:46.363+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:46.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:46.363+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:46.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:46.363+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:46.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:46.363+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:47.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 msgr2=0x7f52c00138e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:47.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00134d0 0x7f52c00138e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:47.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0012f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:47.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c000cfe0 0x7f52c0012f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:47.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:47.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:47.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0013e50 2023-12-04T05:49:47.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:49:47.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0013e50 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:47.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:47.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:47.960+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:48.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:48.161+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0013e50 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:48.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:48.161+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:48.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:48.161+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:48.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:48.562+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0013e50 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:48.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:48.562+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:48.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:48.562+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:49.243 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:49:49.247 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:49:49.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:49.363+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0013e50 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:49.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:49.363+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:49.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:49.363+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:50.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:50.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:50.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0013e50 legacy=0x7f52c00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:50.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:50.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:50.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:50.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c000cb60 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c00151e0 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:50.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:50.961+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.162+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.162+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.162+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.162+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.162+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:51.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.563+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:51.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.563+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:51.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.563+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:51.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.563+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:51.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:51.563+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:52.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:52.364+0000 7f52d8cfe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:52.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:52.364+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:52.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:52.364+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:52.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:52.364+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:52.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:52.364+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:53.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0013e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0014390 2023-12-04T05:49:53.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:53.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0013e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:53.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014390 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:53.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:53.961+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0013e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:54.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:54.162+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014390 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:54.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:54.162+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0013e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:54.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:54.162+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0013e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:54.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:54.563+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0013e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:54.563+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014390 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:54.563+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0013e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:55.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:55.364+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0013e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:55.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:55.364+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014390 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:49:55.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:55.364+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0013e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0013e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0013e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014390 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:56.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c0006d00 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cb60 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:56.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:56.961+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:57.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.162+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:57.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.162+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:57.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.162+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:57.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.162+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:57.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.162+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:49:57.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.563+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:57.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.563+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:57.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.563+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:57.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.563+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:57.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:57.563+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:49:58.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:58.364+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:49:58.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:58.364+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:49:58.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:58.364+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:49:58.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:58.364+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:58.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:58.364+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:49:59.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:59.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0014ce0 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:49:59.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:49:59.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:49:59.962+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:00.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:00.163+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:00.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:00.163+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:00.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:00.163+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:00.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:00.564+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:00.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:00.564+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:00.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:00.564+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:01.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:01.365+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:01.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:01.365+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:01.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:01.365+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:02.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:02.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:02.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:02.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:50:02.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:02.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:02.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c0014ce0 2023-12-04T05:50:02.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:50:02.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c0006d00 2023-12-04T05:50:02.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:02.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:02.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:02.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:02.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:02.962+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.163+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.163+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.163+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.163+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:03.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.163+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:03.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.564+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.564+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.564+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.564+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:03.564+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:04.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:04.365+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:04.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:04.365+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:04.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:04.365+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:04.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:04.365+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:04.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:04.365+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:05.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:05.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:05.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:05.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:05.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:05.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:05.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c000cb60 2023-12-04T05:50:05.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:50:05.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:05.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:05.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:05.962+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:06.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:06.163+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:06.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:06.163+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:06.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:06.163+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:06.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:06.564+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:06.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:06.564+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:06.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:06.564+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:07.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:07.365+0000 7f52d37fe640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:07.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:07.365+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:07.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:07.365+0000 7f52d37fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:08.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c000cfe0 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:50:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cb60 2023-12-04T05:50:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:08.963+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:09.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.164+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:09.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.164+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:09.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.164+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:09.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.164+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:09.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.164+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:09.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.565+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.565+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.565+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.565+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:09.565+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:10.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:10.366+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:10.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:10.366+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:10.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:10.366+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:10.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:10.366+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:10.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:10.366+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:11.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:11.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:11.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:11.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c00133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c00151e0 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0012f90 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c00133a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:11.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:11.963+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c00133a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:12.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:12.164+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c00133a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:12.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:12.164+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:12.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:12.164+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c00133a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:12.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:12.565+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c00133a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:12.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:12.565+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:12.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:12.565+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c00133a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:13.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:13.367+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:13.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:13.367+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c00133a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:13.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:13.367+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c00133a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:14.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 msgr2=0x7f52c00133a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:14.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c0012f90 0x7f52c00133a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:14.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:14.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:50:14.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:14.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:14.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c0006d00 2023-12-04T05:50:14.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cfe0 2023-12-04T05:50:14.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cb60 2023-12-04T05:50:14.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:14.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:14.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:14.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:14.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:14.963+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:15.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.164+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:15.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.164+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:15.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.164+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:15.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.164+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:15.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.164+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:15.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.565+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:15.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.565+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:15.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.565+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:15.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.565+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:15.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:15.565+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:16.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:16.366+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:16.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:16.366+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:16.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:16.366+0000 7f52d8cfe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:16.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:16.366+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:16.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:16.366+0000 7f52d8cfe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 msgr2=0x7f52c0007350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cfe0 0x7f52c0007350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:17.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0014ce0 2023-12-04T05:50:17.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:50:17.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:17.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:17.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:17.964+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:18.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:18.165+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:18.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:18.165+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:18.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:18.165+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:18.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:18.566+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:18.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:18.566+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:18.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:18.566+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:19.315 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:50:19.318 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:50:19.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:19.366+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:19.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:19.366+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:19.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:19.366+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:20.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:20.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0014ce0 legacy=0x7f52c00156b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c000cb60 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:50:20.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cfe0 2023-12-04T05:50:20.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:20.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:20.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:20.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:20.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:20.963+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:21.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.164+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:21.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.164+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:21.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.164+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:21.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.164+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:21.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.164+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:21.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.565+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:21.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.565+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:21.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.565+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:21.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.565+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:21.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:21.565+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:22.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:22.366+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:22.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:22.366+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:22.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:22.366+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:22.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:22.366+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:22.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:22.366+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:23.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:23.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c000cb60 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0011f80 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0011f80 legacy=0x7f52c0012390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:23.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:23.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:23.963+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:24.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:24.164+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0011f80 legacy=0x7f52c0012390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:24.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:24.164+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:24.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:24.164+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:24.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:24.565+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0011f80 legacy=0x7f52c0012390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:24.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:24.565+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:24.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:24.565+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:25.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:25.366+0000 7f52d37fe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0011f80 legacy=0x7f52c0012390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:25.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:25.366+0000 7f52d3fff640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:25.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:25.366+0000 7f52d3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:26.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.963+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:26.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:26.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.963+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c0011f80 legacy=0x7f52c0012390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:26.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.963+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:50:26.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.963+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c00147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:26.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c000cfe0 2023-12-04T05:50:26.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4162af0 con 0x7f52c0006d00 2023-12-04T05:50:26.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c000cb60 2023-12-04T05:50:26.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c0012360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:26.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:26.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:26.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:26.964+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:27.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.165+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c0012360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:27.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.165+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:27.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.165+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:27.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.165+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:27.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.566+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c0012360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:27.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.566+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:27.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.566+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:27.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:27.566+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:28.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:28.367+0000 7f52d8cfe640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c0012360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:28.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:28.367+0000 7f52d3fff640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:28.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:28.367+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:28.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:28.367+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:29.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:29.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:29.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0006d00 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:29.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c000cfe0 legacy=0x7f52c0012360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:29.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 0x7f52c0007240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c00151e0 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c0013a80 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:29.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:29.964+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:30.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.165+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:30.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.165+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:30.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.165+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:30.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.165+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:30.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.566+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:30.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.566+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:30.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.566+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:30.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:30.566+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:31.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:31.367+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:31.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:31.367+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:31.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:31.367+0000 7f52d37fe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:31.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:31.367+0000 7f52d37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:32.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c0013a80 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c00151e0 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d8cfe640 1 -- reap_dead start 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d41628e0 con 0x7f52c0006d00 2023-12-04T05:50:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0008ca0 con 0x7f52c000cb60 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d1ffb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0006670 con 0x7f52c000cfe0 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:32.964+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-04T05:50:33.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.165+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.165+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.165+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.165+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.165+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.165+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-04T05:50:33.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.566+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:33.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.566+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:33.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.566+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:33.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.566+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:33.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.566+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:33.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:33.566+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-04T05:50:34.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:34.367+0000 7f52d3fff640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.111:3300/0 2023-12-04T05:50:34.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:34.367+0000 7f52d3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:34.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:34.367+0000 7f52d37fe640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3300/0 2023-12-04T05:50:34.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:34.367+0000 7f52d8cfe640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.192:3301/0 2023-12-04T05:50:34.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:34.367+0000 7f52d37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:34.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:34.367+0000 7f52d8cfe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-04T05:50:35.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 -- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 msgr2=0x7f52c0007240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52c000cfe0 0x7f52c0007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 -- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 msgr2=0x7f52c0007fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52c000cb60 0x7f52c0007fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 -- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 msgr2=0x7f52c00147a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52c0006d00 0x7f52c00147a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b2090 con 0x7f52c00151e0 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d1ffb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b29c0 con 0x7f52c0012360 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:35.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:35.965+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0012360 legacy=0x7f52c0012770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:36.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:36.166+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:36.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:36.166+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0012360 legacy=0x7f52c0012770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:36.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:36.567+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:36.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:36.567+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0012360 legacy=0x7f52c0012770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:37.368+0000 7f52d3fff640 1 -- >> v1:172.21.15.111:6789/0 conn(0x7f52c00151e0 legacy=0x7f52c000d7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.111:6789/0 2023-12-04T05:50:37.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-04T05:50:37.368+0000 7f52d37fe640 1 -- >> v1:172.21.15.192:6789/0 conn(0x7f52c0012360 legacy=0x7f52c0012770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.192:6789/0 2023-12-04T05:50:38.887 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-12-04T05:50:38.888 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-12-04T05:50:38.904 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-04T05:50:38.904 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-04T05:50:38.904 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:38.907 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-12-04T05:50:38.917 INFO:tasks.ceph.mds.a:Stopped 2023-12-04T05:50:38.918 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-04T05:50:38.918 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.918 INFO:tasks.ceph.mds.c:Stopped 2023-12-04T05:50:38.918 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-04T05:50:38.918 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:38.919 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi192 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-12-04T05:50:38.928 INFO:tasks.ceph.mds.b:Stopped 2023-12-04T05:50:38.929 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-04T05:50:38.929 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:38.929 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi192 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2023-12-04T05:50:38.940 INFO:tasks.ceph.mds.d:Stopped 2023-12-04T05:50:38.940 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-04T05:50:38.941 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-04T05:50:38.941 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.941 INFO:tasks.ceph.osd.0:Stopped 2023-12-04T05:50:38.941 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-04T05:50:38.941 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.941 INFO:tasks.ceph.osd.1:Stopped 2023-12-04T05:50:38.942 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-04T05:50:38.942 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.942 INFO:tasks.ceph.osd.2:Stopped 2023-12-04T05:50:38.942 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-04T05:50:38.942 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.942 INFO:tasks.ceph.osd.3:Stopped 2023-12-04T05:50:38.943 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-04T05:50:38.943 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.943 INFO:tasks.ceph.osd.4:Stopped 2023-12-04T05:50:38.943 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-04T05:50:38.943 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.943 INFO:tasks.ceph.osd.5:Stopped 2023-12-04T05:50:38.943 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-04T05:50:38.943 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.944 INFO:tasks.ceph.osd.6:Stopped 2023-12-04T05:50:38.944 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-04T05:50:38.944 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.944 INFO:tasks.ceph.osd.7:Stopped 2023-12-04T05:50:38.944 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-04T05:50:38.945 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-04T05:50:38.945 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.945 INFO:tasks.ceph.mgr.y:Stopped 2023-12-04T05:50:38.945 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-04T05:50:38.945 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:38.945 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi192 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-12-04T05:50:38.956 INFO:tasks.ceph.mgr.x:Stopped 2023-12-04T05:50:38.956 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-04T05:50:38.956 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-04T05:50:38.956 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.957 INFO:tasks.ceph.mon.a:Stopped 2023-12-04T05:50:38.957 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-04T05:50:38.957 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.957 INFO:tasks.ceph.mon.b:Stopped 2023-12-04T05:50:38.957 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-04T05:50:38.957 INFO:teuthology.orchestra.run:waiting for 300 2023-12-04T05:50:38.960 INFO:tasks.ceph.mon.c:Stopped 2023-12-04T05:50:38.960 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-04T05:50:38.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'client session with non-allowable root' | head -n 1 2023-12-04T05:50:39.059 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:50:39.059 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-04T05:50:39.181 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:50:39.182 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-04T05:50:39.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:50:39.263 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-04T05:50:39.321 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:50:39.322 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-04T05:50:39.379 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:50:39.379 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-04T05:50:39.529 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:50:39.529 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-04T05:50:39.590 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:50:39.591 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-04T05:50:39.647 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:50:39.648 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-04T05:50:39.706 INFO:tasks.ceph:Archiving mon data... 2023-12-04T05:50:39.707 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/data/mon.a.tgz 2023-12-04T05:50:39.707 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-12-04T05:50:39.723 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.GZbCtlTd2Y 2023-12-04T05:50:39.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.GZbCtlTd2Y 2023-12-04T05:50:39.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.GZbCtlTd2Y 2023-12-04T05:50:39.966 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.GZbCtlTd2Y is 383KB 2023-12-04T05:50:40.063 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.GZbCtlTd2Y 2023-12-04T05:50:40.077 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/data/mon.b.tgz 2023-12-04T05:50:40.078 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-04T05:50:40.094 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.tDyr6mkNdx 2023-12-04T05:50:40.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.tDyr6mkNdx 2023-12-04T05:50:40.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.tDyr6mkNdx 2023-12-04T05:50:40.361 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.tDyr6mkNdx is 506KB 2023-12-04T05:50:40.458 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.tDyr6mkNdx 2023-12-04T05:50:40.472 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/data/mon.c.tgz 2023-12-04T05:50:40.473 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-04T05:50:40.527 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.uznEEZjDdJ 2023-12-04T05:50:40.527 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.uznEEZjDdJ 2023-12-04T05:50:40.694 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.uznEEZjDdJ 2023-12-04T05:50:40.781 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.uznEEZjDdJ is 506KB 2023-12-04T05:50:40.885 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.uznEEZjDdJ 2023-12-04T05:50:40.900 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-04T05:50:40.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-04T05:50:40.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-04T05:50:40.968 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-12-04T05:50:40.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-04T05:50:40.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-04T05:50:41.002 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-04T05:50:41.037 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-04T05:50:41.041 INFO:tasks.ceph:Archiving crash dumps... 2023-12-04T05:50:41.042 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/remote/smithi111/crash 2023-12-04T05:50:41.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-04T05:50:41.083 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/remote/smithi192/crash 2023-12-04T05:50:41.084 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-04T05:50:41.128 INFO:tasks.ceph:Shutting down logrotate 2023-12-04T05:50:41.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:50:41.132 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-04T05:50:41.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-04T05:50:41.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-04T05:50:41.269 INFO:tasks.ceph:Compressing logs... 2023-12-04T05:50:41.269 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-04T05:50:41.274 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-04T05:50:41.371 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-12-04T05:50:41.371 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89537.log 2023-12-04T05:50:41.371 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-04T05:50:41.371 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89537.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89537.log.gz 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89697.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.96468.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89697.log.gz 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89457.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95322.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.1.96468.log: /var/log/ceph/ceph-client.admin.89457.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-12-04T05:50:41.372 INFO:teuthology.orchestra.run.smithi192.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89457.log.gz/var/log/ceph/ceph-client.1.95322.log: 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 89.6% -- replaced with /var/log/ceph/ceph-client.1.96468.log.gz 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr: 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 89.6% -- replaced with /var/log/ceph/ceph-client.1.95322.log.gz 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr: --verbose -- /var/log/ceph/ceph-client.admin.89577.log 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89417.log 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89577.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89617.log 2023-12-04T05:50:41.373 INFO:teuthology.orchestra.run.smithi192.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89577.log.gz 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89417.log.gz 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: 90.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97146.log 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89497.log 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr: 88.4%/var/log/ceph/ceph-client.1.97146.log: -- replaced with /var/log/ceph/ceph-client.admin.89617.log.gz 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89497.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-12-04T05:50:41.374 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89497.log.gz 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr: --verbose -- /var/log/ceph/ceph-client.admin.89657.log 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.95757.log 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.89657.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89657.log.gz 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.3.95757.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.d.log: 90.3% -- replaced with /var/log/ceph/ceph-client.3.95757.log.gz 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.c.log: 96.4% -- replaced with /var/log/ceph/ceph-client.1.97146.log.gz 2023-12-04T05:50:41.375 INFO:teuthology.orchestra.run.smithi192.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-12-04T05:50:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106438.log 2023-12-04T05:50:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108563.log 2023-12-04T05:50:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:gzip/var/log/ceph/ceph-client.admin.106438.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.103999.log 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.108563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99777.log 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106438.log.gz 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8%/var/log/ceph/ceph-client.admin.103999.log: -- replaced with /var/log/ceph/ceph-client.admin.108563.log.gz 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102909.log 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103999.log.gz 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94921.log 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97093.log 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102909.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99777.log.gz 2023-12-04T05:50:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.94921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100803.log 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102909.log.gz 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97093.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94921.log.gz 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100803.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.97093.log.gz --verbose 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.99094.log 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100803.log.gz 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108605.log 2023-12-04T05:50:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88730.log 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.108605.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.99094.log.gz 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95087.log 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.88730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102177.log 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.88730.log.gz 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95087.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108605.log.gz 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106625.log 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95087.log.gz 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101191.log 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106625.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102177.log.gz 2023-12-04T05:50:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109363.log 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106625.log.gz/var/log/ceph/ceph-client.admin.101191.log: 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104781.log 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109363.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.101191.log.gz 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102135.log 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104781.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109363.log.gz 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106396.log 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104781.log.gz 2023-12-04T05:50:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102550.log 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106396.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.102135.log.gz --verbose 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.97261.log 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98049.log 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106396.log.gz 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97261.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102550.log.gz 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103517.log 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97261.log.gz 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107566.log 2023-12-04T05:50:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.0.98049.log: /var/log/ceph/ceph-client.admin.103517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97615.log 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96645.log 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97615.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107566.log.gz 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.103517.log.gz 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96643.log 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.0.98049.log.gz 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96645.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97615.log.gz 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106312.log 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96645.log.gz 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96967.log 2023-12-04T05:50:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106312.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96643.log.gz 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82656.log 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96967.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106312.log.gz 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102771.log 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph.tmp-client.admin.82656.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.101485.log -- replaced with /var/log/ceph/ceph-client.admin.96967.log.gz 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82656.log.gz 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105053.log 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96925.log 2023-12-04T05:50:41.453 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102771.log.gz 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105053.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101485.log.gz 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107218.log 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105053.log.gz 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107820.log 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107218.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.96925.log.gz -5 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100435.log 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107218.log.gz 2023-12-04T05:50:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101990.log 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105312.log 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107820.log.gz 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101990.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100435.log.gz 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109709.log 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101990.log.gz 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.0.105312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94825.log 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104655.log 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109709.log.gz 2023-12-04T05:50:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.94825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110217.log 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.0.105312.log.gz 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101071.log 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104655.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94825.log.gz 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99388.log 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104655.log.gz 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101071.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.108470.log -- replaced with /var/log/ceph/ceph-client.admin.110217.log.gz 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99388.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101071.log.gz 2023-12-04T05:50:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103163.log 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9%/var/log/ceph/ceph-client.admin.108470.log: -- replaced with /var/log/ceph/ceph-client.admin.99388.log.gz 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110930.log 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108470.log.gz 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100351.log 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110930.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103163.log.gz -5 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105589.log 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100267.log 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100351.log.gz 2023-12-04T05:50:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103079.log 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105589.log.gz 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100603.log 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100267.log: /var/log/ceph/ceph-client.admin.103079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95411.log 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100267.log.gz 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100603.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103079.log.gz 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106877.log 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100603.log.gz 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107386.log 2023-12-04T05:50:41.458 INFO:teuthology.orchestra.run.smithi111.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.110930.log.gz 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95411.log.gz 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106751.log 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106877.log: /var/log/ceph/ceph-client.admin.107386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103289.log 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106877.log.gz 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103121.log 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107386.log.gz 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103289.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106751.log.gz 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99945.log 2023-12-04T05:50:41.459 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103289.log.gz 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104697.log 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103391.log 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103121.log.gz 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99945.log.gz 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96346.log 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103391.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104697.log.gz 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97135.log 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96346.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.103391.log.gz 2023-12-04T05:50:41.460 INFO:teuthology.orchestra.run.smithi111.stderr: /var/log/ceph/ceph-client.admin.102815.log 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96346.log.gz/var/log/ceph/ceph-client.admin.97135.log: 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109447.log 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102815.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97135.log.gz 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110387.log 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102815.log.gz 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104613.log 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101233.log 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109447.log.gz 2023-12-04T05:50:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104613.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110387.log.gz 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97345.log 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104613.log.gz 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110617.log 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101905.log 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101233.log.gz 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110617.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97345.log.gz 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100105.log 2023-12-04T05:50:41.462 INFO:teuthology.orchestra.run.smithi111.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110617.log.gz 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96350.log 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96347.log 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101905.log.gz 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96350.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100105.log.gz 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102951.log 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96350.log.gz 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110281.log 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105631.log 2023-12-04T05:50:41.463 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96347.log.gz 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110281.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102951.log.gz 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106270.log 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110281.log.gz 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104225.log 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106793.log 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105631.log.gz 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104225.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106270.log.gz 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104325.log 2023-12-04T05:50:41.464 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104225.log.gz 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96647.log 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101113.log 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106793.log.gz 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96647.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104325.log.gz 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104451.log 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96647.log.gz 2023-12-04T05:50:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99514.log 2023-12-04T05:50:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104451.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.101113.log.gz --verbose 2023-12-04T05:50:41.466 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.95495.log 2023-12-04T05:50:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99514.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104451.log.gz 2023-12-04T05:50:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99556.log 2023-12-04T05:50:41.467 INFO:teuthology.orchestra.run.smithi111.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99514.log.gz 2023-12-04T05:50:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103433.log 2023-12-04T05:50:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99640.log 2023-12-04T05:50:41.467 INFO:teuthology.orchestra.run.smithi111.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95495.log.gz 2023-12-04T05:50:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103433.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99556.log.gz 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99010.log 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103433.log.gz 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95255.log 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100393.log 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99640.log.gz 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95255.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99010.log.gz 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104571.log 2023-12-04T05:50:41.468 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95255.log.gz 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106835.log 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104571.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.100393.log.gz --verbose 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.107862.log 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106835.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.104571.log.gz 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97513.log 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106835.log.gz 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96013.log 2023-12-04T05:50:41.469 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107862.log.gz 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106961.log 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110701.log 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96013.log.gz 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106582.log 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.97513.log.gz 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104907.log 2023-12-04T05:50:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110701.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106961.log.gz 2023-12-04T05:50:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99220.log 2023-12-04T05:50:41.471 INFO:teuthology.orchestra.run.smithi111.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106582.log.gz 2023-12-04T05:50:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103475.log 2023-12-04T05:50:41.471 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104907.log.gz 2023-12-04T05:50:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105673.log 2023-12-04T05:50:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99220.log: /var/log/ceph/ceph-client.admin.103475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110365.log 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.110701.log.gz 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99220.log.gz 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105673.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103475.log.gz 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104099.log 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110365.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105673.log.gz 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95665.log 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104099.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110365.log.gz 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106186.log 2023-12-04T05:50:41.472 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104099.log.gz 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104141.log 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106186.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.95665.log.gz -5 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107524.log 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106186.log.gz/var/log/ceph/ceph-client.admin.104141.log: 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97219.log 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104141.log.gz 2023-12-04T05:50:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96011.log 2023-12-04T05:50:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107524.log: /var/log/ceph/ceph-client.admin.97219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109793.log 2023-12-04T05:50:41.475 INFO:teuthology.orchestra.run.smithi111.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.107524.log.gz 2023-12-04T05:50:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109489.log 2023-12-04T05:50:41.475 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97219.log.gz 2023-12-04T05:50:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109793.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96011.log.gz 2023-12-04T05:50:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97471.log 2023-12-04T05:50:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109489.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109793.log.gz 2023-12-04T05:50:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110533.log 2023-12-04T05:50:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97471.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109489.log.gz 2023-12-04T05:50:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109961.log 2023-12-04T05:50:41.476 INFO:teuthology.orchestra.run.smithi111.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97471.log.gz 2023-12-04T05:50:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110533.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-04T05:50:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109961.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110533.log.gz 2023-12-04T05:50:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107056.log 2023-12-04T05:50:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph.audit.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109961.log.gz 2023-12-04T05:50:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95044.log 2023-12-04T05:50:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99178.log 2023-12-04T05:50:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101275.log 2023-12-04T05:50:41.477 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107056.log.gz 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101527.log 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95044.log.gz 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101275.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99178.log.gz 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98154.log 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101275.log.gz/var/log/ceph/ceph-client.admin.101527.log: 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95623.log 2023-12-04T05:50:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.98154.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101527.log.gz 2023-12-04T05:50:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106709.log 2023-12-04T05:50:41.479 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98154.log.gz 2023-12-04T05:50:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100761.log 2023-12-04T05:50:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106709.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.107136.log 2023-12-04T05:50:41.479 INFO:teuthology.orchestra.run.smithi111.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95623.log.gz 2023-12-04T05:50:41.479 INFO:teuthology.orchestra.run.smithi111.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106709.log.gz 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98884.log 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100761.log.gz 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103205.log 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.98884.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.107136.log.gz --verbose 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.99346.log 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103205.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.98884.log.gz 2023-12-04T05:50:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99598.log 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99346.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103205.log.gz 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96687.log 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99346.log.gz 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96008.log 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99598.log.gz 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105815.log 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96008.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96687.log.gz 2023-12-04T05:50:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96686.log 2023-12-04T05:50:41.482 INFO:teuthology.orchestra.run.smithi111.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96008.log.gz 2023-12-04T05:50:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105773.log 2023-12-04T05:50:41.482 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105815.log.gz 2023-12-04T05:50:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5/var/log/ceph/ceph-client.admin.96686.log: --verbose -- /var/log/ceph/ceph-client.admin.107430.log 2023-12-04T05:50:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99136.log 2023-12-04T05:50:41.482 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96686.log.gz 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107430.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105773.log.gz 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96883.log 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107430.log.gz 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109279.log 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96883.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.99136.log.gz -5 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110003.log 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109279.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96883.log.gz 2023-12-04T05:50:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106522.log 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109279.log.gz 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107260.log 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110003.log.gz 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101695.log 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107260.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106522.log.gz 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102729.log 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101695.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107260.log.gz 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100309.log 2023-12-04T05:50:41.484 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101695.log.gz 2023-12-04T05:50:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102644.log 2023-12-04T05:50:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109751.log 2023-12-04T05:50:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100309.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102729.log.gz 2023-12-04T05:50:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100945.log 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100309.log.gz 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102644.log.gz 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108012.log 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109751.log.gz 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101401.log 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.108012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100887.log 2023-12-04T05:50:41.486 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100945.log.gz 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108012.log.gz 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107694.log 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100887.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101401.log.gz 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106228.log 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107694.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100887.log.gz 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100987.log 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5%/var/log/ceph/ceph-client.admin.106228.log: -- replaced with /var/log/ceph/ceph-client.admin.107694.log.gz 2023-12-04T05:50:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96015.log 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106228.log.gz 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:gzip/var/log/ceph/ceph-client.admin.100987.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.102859.log 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99903.log 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100987.log.gz 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102859.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96015.log.gz 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95129.log 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102859.log.gz 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95791.log 2023-12-04T05:50:41.488 INFO:teuthology.orchestra.run.smithi111.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99903.log.gz 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95129.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96338.log 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95129.log.gz 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95791.log.gz 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98800.log 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104409.log 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96338.log.gz 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.98800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100147.log 2023-12-04T05:50:41.489 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.98800.log.gz -5 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105857.log 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104409.log: /var/log/ceph/ceph-client.admin.100147.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104409.log.gz 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98926.log 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100147.log.gz 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110449.log 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105857.log: /var/log/ceph/ceph-client.admin.98926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96342.log 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105857.log.gz 2023-12-04T05:50:41.490 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98926.log.gz 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109175.log 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108271.log 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96342.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110449.log.gz 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94733.log 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96342.log.gz 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.0.108271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107736.log 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109175.log.gz 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.94733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100845.log 2023-12-04T05:50:41.491 INFO:teuthology.orchestra.run.smithi111.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94733.log.gz 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95749.log 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98690.log 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107736.log.gz 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100845.log.gz 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95213.log 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.98690.log: gzip -5 --verbose 88.3% -- /var/log/ceph/ceph-client.admin.103583.log -- replaced with /var/log/ceph/ceph-client.admin.95749.log.gz 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95213.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98690.log.gz 2023-12-04T05:50:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102993.log 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95213.log.gz 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97303.log 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101359.log 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102993.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103583.log.gz 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97303.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102993.log.gz 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.102338.log 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97303.log.gz 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100021.log 2023-12-04T05:50:41.493 INFO:teuthology.orchestra.run.smithi111.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101359.log.gz 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:gzip/var/log/ceph/ceph-client.admin.102338.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.104367.log 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100021.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102338.log.gz 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110471.log 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100021.log.gz 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103349.log 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104367.log.gz 2023-12-04T05:50:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95833.log 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103349.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110471.log.gz 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109877.log 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103349.log.gz 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96014.log 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95833.log.gz 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103842.log 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109631.log 2023-12-04T05:50:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96014.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109877.log.gz 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.0.103842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95453.log 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96014.log.gz 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107474.log 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95453.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109631.log.gz 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.0.103842.log.gz 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100189.log 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95453.log.gz 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107778.log 2023-12-04T05:50:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107474.log: /var/log/ceph/ceph-client.admin.100189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104865.log 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107474.log.gz 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100189.log.gz 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107778.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107608.log 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104865.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107778.log.gz 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104865.log.gz 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.99819.log 94.7% -- replaced with /var/log/ceph/ceph-client.0.108271.log.gz 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107608.log: gzip -5 --verbose 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107608.log.gz 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr: -- /var/log/ceph/ceph-client.admin.109321.log 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108671.log 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0%/var/log/ceph/ceph-client.admin.109321.log: -- replaced with /var/log/ceph/ceph-client.admin.99819.log.gz 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99304.log 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.108671.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109321.log.gz 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106919.log 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101569.log 2023-12-04T05:50:41.498 INFO:teuthology.orchestra.run.smithi111.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.108671.log.gz 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106919.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99304.log.gz 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.98526.log 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106919.log.gz 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102687.log 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.2.98526.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101569.log.gz 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97429.log 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102465.log 2023-12-04T05:50:41.499 INFO:teuthology.orchestra.run.smithi111.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102687.log.gz 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97429.log: 90.8% -- replaced with /var/log/ceph/ceph-client.2.98526.log.gz 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99052.log 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.97429.log.gz 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95299.log 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102465.log: /var/log/ceph/ceph-client.admin.99052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100645.log 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102465.log.gz 2023-12-04T05:50:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95299.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99052.log.gz 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104739.log 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95299.log.gz 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101317.log 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104739.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100645.log.gz 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100063.log 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101317.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104739.log.gz 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104041.log 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100063.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101317.log.gz 2023-12-04T05:50:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104267.log 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100063.log.gz 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97051.log 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96010.log 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104267.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104041.log.gz 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98968.log 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104267.log.gz 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95581.log 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97051.log.gz 2023-12-04T05:50:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96010.log: /var/log/ceph/ceph-client.admin.98968.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96010.log.gz 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109919.log 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95581.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98968.log.gz 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108520.log 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109919.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95581.log.gz 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97177.log 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.108520.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.109919.log.gz -5 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102257.log 2023-12-04T05:50:41.503 INFO:teuthology.orchestra.run.smithi111.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108520.log.gz 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101863.log 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97387.log 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102257.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97177.log.gz 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105505.log/var/log/ceph/ceph-client.admin.101863.log: 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102257.log.gz 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97387.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101863.log.gz 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96351.log 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97387.log.gz 2023-12-04T05:50:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96012.log 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105505.log.gz 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101948.log 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96012.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96351.log.gz 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109573.log 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96012.log.gz 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110659.log 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101948.log.gz 2023-12-04T05:50:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106144.log 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110659.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109573.log.gz 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110659.log.gz 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101653.log 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106144.log.gz/var/log/ceph/ceph-mgr.y.log: 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104949.log 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110045.log 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104949.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101653.log.gz 2023-12-04T05:50:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109531.log 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110045.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104949.log.gz 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99472.log 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110045.log.gz 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96642.log 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109531.log.gz 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106354.log 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96642.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99472.log.gz 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107344.log 2023-12-04T05:50:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106354.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96642.log.gz 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109835.log 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106354.log.gz 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103037.log 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109835.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95002.log 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107344.log.gz 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107302.log 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109835.log.gz 2023-12-04T05:50:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103037.log.gz 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95002.log: --verbose -- /var/log/ceph/ceph-client.admin.107652.log 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107302.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102032.log 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95002.log.gz 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107302.log.gz 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106480.log 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4%/var/log/ceph/ceph-client.admin.102032.log: -- replaced with /var/log/ceph/ceph-client.admin.107652.log.gz 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102380.log 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102032.log.gz 2023-12-04T05:50:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89103.log 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102380.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106480.log.gzgzip 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99430.log 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.89103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96348.log 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102380.log.gz 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89103.log.gz 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99430.log: --verbose -- /var/log/ceph/ceph-client.admin.101737.log 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96348.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99430.log.gz 2023-12-04T05:50:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.96644.log -- replaced with /var/log/ceph/ceph-client.admin.96348.log.gz 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101779.log 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96644.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101737.log.gz 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96644.log.gz 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104823.log 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101779.log.gz 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105547.log 2023-12-04T05:50:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95361.log 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104823.log.gz 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99682.log 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4%/var/log/ceph/ceph-client.admin.95361.log: -- replaced with /var/log/ceph/ceph-client.admin.105547.log.gz 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96646.log 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95361.log.gz 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110302.log 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99682.log: /var/log/ceph/ceph-client.admin.96646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106102.log 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110302.log: 89.1% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99682.log.gz -- replaced with /var/log/ceph/ceph-client.admin.96646.log.gz 2023-12-04T05:50:41.512 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107904.log 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110302.log.gz 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104493.log 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106102.log: /var/log/ceph/ceph-client.admin.107904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95539.log 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104493.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106102.log.gz 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107904.log.gz 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100477.log 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104493.log.gz 2023-12-04T05:50:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96009.log 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95539.log: /var/log/ceph/ceph-client.admin.100477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109237.log 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96009.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95539.log.gz 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100477.log.gz 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100519.log 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96009.log.gz 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.109237.log: -- /var/log/ceph/ceph.log 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100519.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102600.log 2023-12-04T05:50:41.514 INFO:teuthology.orchestra.run.smithi111.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109237.log.gz 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100519.log.gz 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101029.log 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101821.log 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101029.log: /var/log/ceph/ceph-client.admin.102600.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102600.log.gz 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101029.log.gz 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-12-04T05:50:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101821.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102422.log 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mds.c.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101821.log.gz 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97009.log 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102422.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102422.log.gz 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99861.log 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.97009.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97009.log.gz 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110575.log 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99861.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99861.log.gz 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103247.log 2023-12-04T05:50:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110575.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110575.log.gz 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105731.log 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.103247.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103247.log.gz 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104183.log 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105731.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105731.log.gz 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96349.log 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104183.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104183.log.gz 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101443.log 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.96349.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96349.log.gz 2023-12-04T05:50:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98648.log 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101443.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101443.log.gz 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108428.log 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.98648.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98648.log.gz 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102092.log 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.108428.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108428.log.gz 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107946.log 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102092.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102092.log.gz 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105899.log 2023-12-04T05:50:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.107946.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.107946.log.gz 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102507.log 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.105899.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105899.log.gz 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110120.log 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.102507.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102507.log.gz 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101611.log 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110120.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110120.log.gz 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95171.log 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.101611.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101611.log.gz 2023-12-04T05:50:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106667.log 2023-12-04T05:50:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95171.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95171.log.gz 2023-12-04T05:50:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109405.log 2023-12-04T05:50:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.106667.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106667.log.gz 2023-12-04T05:50:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100561.log 2023-12-04T05:50:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.109405.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109405.log.gz 2023-12-04T05:50:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99262.log 2023-12-04T05:50:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100561.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100561.log.gz 2023-12-04T05:50:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95707.log 2023-12-04T05:50:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.99262.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99262.log.gz 2023-12-04T05:50:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89001.log 2023-12-04T05:50:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.95707.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95707.log.gz 2023-12-04T05:50:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104991.log 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.89001.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89001.log.gz 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.104991.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104991.log.gz 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100719.log 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98842.log 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.100719.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100719.log.gz 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110216.log 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.98842.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98842.log.gz 2023-12-04T05:50:41.523 INFO:teuthology.orchestra.run.smithi111.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-12-04T05:50:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/ceph-client.admin.110216.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110216.log.gz 2023-12-04T05:50:41.524 INFO:teuthology.orchestra.run.smithi192.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-12-04T05:50:41.524 INFO:teuthology.orchestra.run.smithi111.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-12-04T05:50:41.525 INFO:teuthology.orchestra.run.smithi111.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-12-04T05:50:41.536 INFO:teuthology.orchestra.run.smithi192.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-12-04T05:50:41.582 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-12-04T05:50:41.582 INFO:teuthology.orchestra.run.smithi111.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-12-04T05:50:41.599 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-12-04T05:50:41.605 INFO:teuthology.orchestra.run.smithi111.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-12-04T05:50:41.608 INFO:teuthology.orchestra.run.smithi192.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-12-04T05:50:41.609 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-12-04T05:50:41.610 INFO:teuthology.orchestra.run.smithi111.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-12-04T05:50:41.611 INFO:teuthology.orchestra.run.smithi192.stderr: 2023-12-04T05:50:41.611 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m0.281s 2023-12-04T05:50:41.611 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m1.319s 2023-12-04T05:50:41.612 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.071s 2023-12-04T05:50:41.666 INFO:teuthology.orchestra.run.smithi111.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-12-04T05:50:41.679 INFO:teuthology.orchestra.run.smithi111.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-12-04T05:50:41.681 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-04T05:50:41.681 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m0.397s 2023-12-04T05:50:41.681 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m1.675s 2023-12-04T05:50:41.681 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.240s 2023-12-04T05:50:41.682 INFO:tasks.ceph:Archiving logs... 2023-12-04T05:50:41.682 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/remote/smithi111/log 2023-12-04T05:50:41.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-04T05:50:42.277 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/remote/smithi192/log 2023-12-04T05:50:42.278 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-04T05:50:42.509 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-12-04T05:50:42.520 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-04T05:50:42.600 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-12-04T05:50:42.606 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-04T05:50:42.606 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-04T05:50:42.614 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-04T05:50:42.690 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-04T05:50:42.700 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-04T05:50:42.700 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-04T05:50:42.703 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-04T05:50:42.715 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2023-12-04T05:50:42.717 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2023-12-04T05:50:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-04T05:50:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-12-04T05:50:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 +845us[ +859us] +/- 80ms 2023-12-04T05:50:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -49us[ -35us] +/- 69ms 2023-12-04T05:50:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -718us[ -704us] +/- 78ms 2023-12-04T05:50:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:50:42.722 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-04T05:50:42.722 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-12-04T05:50:42.722 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +846us[ +846us] +/- 80ms 2023-12-04T05:50:42.722 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -36us[+3971ns] +/- 69ms 2023-12-04T05:50:42.722 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -724us[ -724us] +/- 78ms 2023-12-04T05:50:42.722 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-04T05:50:42.723 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-04T05:50:42.742 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-04T05:50:42.742 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-04T05:50:42.776 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-04T05:50:42.880 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-04T05:50:42.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-04T05:50:43.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:43.012 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-04T05:50:43.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:43.044 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-04T05:50:43.055 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-04T05:50:43.086 INFO:teuthology.task.internal:Duration was 1722.044461 seconds 2023-12-04T05:50:43.086 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-04T05:50:43.131 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-04T05:50:43.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-04T05:50:43.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-04T05:50:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-04T05:50:43.180 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-04T05:50:43.656 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-04T05:50:43.656 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2023-12-04T05:50:43.657 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-04T05:50:43.677 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2023-12-04T05:50:43.677 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-04T05:50:43.722 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-04T05:50:43.722 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-04T05:50:43.725 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-04T05:50:43.791 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-04T05:50:43.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-04T05:50:43.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-04T05:50:43.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-04T05:50:43.946 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-04T05:50:43.946 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-04T05:50:43.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-04T05:50:44.000 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-04T05:50:44.013 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-04T05:50:44.016 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-04T05:50:44.040 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2023-12-04T05:50:44.065 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2023-12-04T05:50:44.087 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-04T05:50:44.112 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi111.front.sepia.ceph.com, flagging run as failed 2023-12-04T05:50:44.112 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-04T05:50:44.142 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi192.front.sepia.ceph.com, flagging run as failed 2023-12-04T05:50:44.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-04T05:50:44.154 INFO:teuthology.task.internal:Transferring archived files... 2023-12-04T05:50:44.155 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/remote/smithi111 2023-12-04T05:50:44.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-04T05:50:51.648 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-12-04T05:50:51.665 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641/remote/smithi111/coredump/1701668485.107137.core.gz: gzip compressed data, was "1701668485.107137.core", last modified: Mon Dec 4 05:50:44 2023, max compression, original size modulo 2^32 235204608 2023-12-04T05:50:51.666 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-12-04T05:50:52.630 INFO:teuthology.task.internal: core looks like: /tmp/tmpzen1e28w: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-12-04T05:50:52.657 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i a 2023-12-04T05:50:52.657 DEBUG:teuthology.orchestra.run.smithi111:> which 'ceph-mds -f --cluster ceph -i a' 2023-12-04T05:50:52.676 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-mds 2023-12-04T05:50:52.676 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph 2023-12-04T05:50:52.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:52.677 INFO:teuthology.orchestra.run.smithi111.stderr:/usr/bin/which: invalid option -- 'f' 2023-12-04T05:50:52.678 INFO:teuthology.orchestra.run.smithi111.stderr:/usr/bin/which: unrecognized option '--cluster' 2023-12-04T05:50:52.678 INFO:teuthology.orchestra.run.smithi111.stderr:/usr/bin/which: no a in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2023-12-04T05:50:52.678 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 1: "which 'ceph-mds -f --cluster ceph -i a'" 2023-12-04T05:50:52.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-04T05:50:52.709 INFO:teuthology.task.internal:Not uploading archives. 2023-12-04T05:50:52.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-04T05:50:52.760 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-04T05:50:52.761 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-04T05:50:52.770 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-04T05:50:52.785 INFO:teuthology.orchestra.run.smithi111.stdout: 265256 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest 2023-12-04T05:50:52.785 INFO:teuthology.orchestra.run.smithi111.stdout: 265267 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest/archive 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265437 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest/archive/syslog 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265312 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Dec 4 05:21 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265314 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Dec 4 05:21 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265315 116 -rw-r--r-- 1 ubuntu ubuntu 117958 Dec 4 05:50 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265292 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest/archive/audit 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265313 284 -rw------- 1 ubuntu ubuntu 289811 Dec 4 05:50 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 265310 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 4 05:41 /home/ubuntu/cephtest/archive/coredump 2023-12-04T05:50:52.786 INFO:teuthology.orchestra.run.smithi111.stdout: 278114 25260 -rw------- 1 root root 235204608 Dec 4 05:41 /home/ubuntu/cephtest/archive/coredump/1701668485.107137.core 2023-12-04T05:50:52.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-04T05:50:52.788 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-12-04T05:50:52.788 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_47c1c547372ed5bff7984647dc200bac0e019caf/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi111 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-12-04T05:50:52.802 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-04T05:50:52.813 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-04T05:50:52.851 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-12-04T05:50:52.851 INFO:teuthology.nuke:Checking targets against current locks 2023-12-04T05:50:52.859 INFO:teuthology.orchestra.run.smithi192.stdout: 265244 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265245 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest/archive 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest/archive/syslog 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265296 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Dec 4 05:22 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265251 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Dec 4 05:22 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265250 108 -rw-r--r-- 1 ubuntu ubuntu 110339 Dec 4 05:50 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 394675 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 4 05:50 /home/ubuntu/cephtest/archive/audit 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 394954 256 -rw------- 1 ubuntu ubuntu 258376 Dec 4 05:50 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265248 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 4 05:42 /home/ubuntu/cephtest/archive/coredump 2023-12-04T05:50:52.860 INFO:teuthology.orchestra.run.smithi192.stdout: 265333 23240 -rw------- 1 root root 250789888 Dec 4 05:41 /home/ubuntu/cephtest/archive/coredump/1701668506.96870.core 2023-12-04T05:50:52.861 INFO:teuthology.orchestra.run.smithi192.stdout: 265334 24612 -rw------- 1 root root 250912768 Dec 4 05:42 /home/ubuntu/cephtest/archive/coredump/1701668526.96871.core 2023-12-04T05:50:52.861 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-12-04T05:50:52.885 DEBUG:teuthology.nuke:shortname: smithi111 2023-12-04T05:50:52.885 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-04T05:50:52.911 DEBUG:teuthology.nuke:shortname: smithi192 2023-12-04T05:50:52.912 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-04T05:50:52.917 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-04 05:08:25.501655', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP7TOh+HZ3oyc1UE1dEmLMraqDwNhhMUnk6LiwOPFMRjk+UfO70Y+SWF1YQSouzAW6QbWiogrnTHE8H7oEPYl2U='} 2023-12-04T05:50:52.940 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-04_05:06:51-fs-wip-vshankar-testing-20231127.102654-distro-default-smithi/7476641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-04 05:08:25.503205', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOQEz4rvWx4sxcwW76PgSTUPMID8HIMA8zxtTk/Wv1482QciO4OcN7Sh7KpO2C9zAeVTY+hosTMmRVjHxmy5GEY='} 2023-12-04T05:50:52.968 INFO:teuthology.orchestra.console.smithi111:Power off 2023-12-04T05:50:52.969 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-04T05:50:52.997 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2023-12-04T05:50:52.997 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-04T05:50:53.015 INFO:teuthology.orchestra.console.smithi192:Power off 2023-12-04T05:50:53.016 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-04T05:50:53.026 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-12-04T05:50:53.039 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-12-04T05:50:53.039 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-04T05:50:53.066 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-04T05:50:57.027 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-04T05:50:57.068 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-04T05:50:57.141 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-12-04T05:50:57.181 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-04T05:51:01.146 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-04T05:51:01.182 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-04T05:51:01.260 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2023-12-04T05:51:01.294 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-12-04T05:51:01.361 INFO:teuthology.orchestra.console.smithi111:Power off completed 2023-12-04T05:51:01.395 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-12-04T05:51:01.500 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} duration: 1722.0444612503052 failure_reason: 'Test failure: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap), test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap)' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f33f8180c7ce4803923693d317b1b22e status: fail success: false 2023-12-04T05:51:01.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-04T05:51:01.580 INFO:teuthology.run:FAIL